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1. SUMMARY 

Concord Scientific Corporation (CSC) has been contracted by 
the Ontario Ministry of the Environment (MOE) to carry out two audits of 
the Acidic Precipitation in Ontario Study (APIOS) monitoring network 
(laboratory activities were not included in the audit since these 
aspects were extensively addressed in a previous audit, (CSC, 1982)). 

This quality assurance audit provides an independent review 
and evaluation of the operating and quality assurance procedures for the 
APIOS monitoring network. As defined in Phase I, the objective of the 
audit is to assess how the various components within the APIOS programme 
are contributing to the production of "high quality" data. 

In general, the Phase II audits indicate that the APIOS 
personnel are properly executing the procedures that are documented in 
the APIOS Technical and Operating Manual (Bardswick, 1983). 

Operators' training for the most part has been upgraded or 
reinforced and operators are following the currently recommended opera- 
ting and handling procedures. This ensures that good quality data are 
collected. 

The majority of the monitoring sites visited during this audit 
met siting criteria and the instruments were operating according to 
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design specifications. Sites which are in violation of the siting 
criteria, or operators not using approved handling procedures and there- 
fore jeopardizing sample integrity, are described 1n detail in the 
performance related findings of the audit report (Section 3.1). A 
special review by the QA Coordinator (QAC) of these specific sites and 
operators should be carried out. 

Since the Phase I audit, the data validation procedures for 
cumulative and daily air concentration data have been documented (Chung, 
1984; Kirk, 1984) and the 1982 daily and cumulative air concentration 
listings have been published (MOE, 1984). These data validation proce- 
dures have been well conceived in general, however, some recormnendations 
with reference to data qualification and reporting have been made 
(Section 4.1). 

As recommended by the QAC, the data handling portion of the 
audit consisted of a performance audit of cumulative and daily air con- 
centration data and a systems audit of sequential sampler methodologies 
and data handling procedures. 

A draft copy of the APIOS QA Manual (CSC, 1984b) has been sub- 
mitted to the APIOS management for review. As described in the Phase I 
audit (CSC, 1984a), this manual brings together all the various quality 
assurance procedures currently being carried out (e.g. co-located sam- 
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pllng, calibration, blind samples, etc.) as well as describing QA acti- 
vities which have not yet been incorporated into the operating system 
{e.g. internal audits, corrective action, procurement control, etc.). 
This manual also decribes how these procedures fit together in the 
quality assurance system, so that estimates of data accuracy, precision, 
completeness, comparability and representativeness can be made. 

Finally, as stated in the Phase I audit, the auditors have 
noticed marked improvements in all areas of the programme, particularly 
in the areas of network documentation, data validation and the implemen- 
tation of additional routine QC procedures. With the completion of the 
QA manual, the full APIOS quality assurance programme can become opera- 
tional, and network data evaluated with respect to the network QA objec- 
tives and the standard QA elements of accuracy, precision, representa- 
tiveness and comparability. 
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2. INTRODUCTION 

Concord Scientific Corporation (CSC) has undertaken on behalf 
of the Ontario Ministry of the Environment (MOE) two audits of its 
Acidic Precipitation in Ontario Study (APIOS) monitoring network (labo- 
ratory activities were not included in the audit since these aspects 
were extensively addressed in a previous audit, (CSC, 1982)). 

The Phase I audit was carried out during the fall of 1983 
through to the winter of 1984 and the Phase II audit was carried out 
during the summer of 1984. The findings of the Phase I audit were 
presented in the Phase I Report (CSC, 1984a] to the APIOS QA 
Coordinator. 

This report contains the findings of the Phase II audit inclu- 
ding recommendations for changes to the APIOS programme. Volume I of 
this report contains four main chapters as did the Phase I Report. 

The first chapter summarizes the audit results and points out 
major strengths and weaknesses of the programme. The third chapter is 
dedicated to performance related audit findings. These findings are a 
quantitative and qualitative measure of how well the APIOS procedures 
are being executed. The fourth chapter reports on system related audit 
findings, and gives details on modifications recommended to aspects of 
the programme which will assist in attainment of both network and 
quality assurance objectives (Bardswick, 1983b) and provide greater 
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efficiency In operations. The last chapter collates the recommendations 
described throughout the report in a summary form, as well as reviews 
the actions on the recommendations from the Phase I audit. 

Volume II of the Phase II Audit Report contains the completed 
Phase II site audit questionnaires prepared by CSC's auditors, including 
photographs of the audited sites. 
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3. PERFORMANCE RELATED AUDIT 

The purpose of a performance audit 1s to quantitatively evalu- 
ate the quality of the data produced by the total measurement system, 
e.g. sample collection, sample analysis and data processing (U.S. EPA 
1977). 

The performance audit carried out in Phase II is broken down 
into three components, site audits, technologist audits, and data hand- 
ling and validation audits. For each of the components the network's 
performance with respect to documented APIOS procedures was described in 
quantitative and qualitative terms. The findings are discussed below 
and the recommendations from this chapter are summarized in Chapter 5. 

3.1 Status of Monitoring Sites 

Site evaluations were carried out at thirteen APIOS monitoring 
stations. The list of sites inspected is presented in Table 3.1. 

The site visits included audits and evaluations of the site, 
operator sample handling procedures, plus configuration and calibration 
checks on instrumentation. 

The questionnaires and audit check lists used during the site 
visits are identical to those used during the Phase I Audit, and are 
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TABLE 3.1 



Sites Visited During the Phase II Site Audits 



Site 



Type 



Merlin 

Port Stanley 

Pickle Lake 

Forbes Township 

Quetico Centre 

Ear Falls 

Experimental Lakes Area 

Dorset 

Whitney 
Col dwater 
Gowganda 
Winisk 
Whitman Creek 



Cumulative Wet 

Cumulative Wet & Dry 

Cumulative Wet & Dry 

Event Wet 

Cumulative Wet and Event Wet 

Cumulative Wet 4 Dry 

Cumulative Wet 

Cumulative Wet & Dry, Event Wet 

A Dry 

Cumulative Wet 

Cumulative Wet 

Cumulative Wet & Dry 

Cumulative Wet 

Event Wet 
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presented in Appendix I. The completed questionnaires, site and aerial 
photographs are presented under a separate cover in Volume II of this 
report. Site audit summaries are presented in Appendix II. These 
summaries include site evaluations, instrument evaluations, operator 
evaluations and corrective action required at each site. 

Table 3.2 summarizes the audit findings for each site. This 
table is broken down into two components, a physical siting related 
component and an operations related component. The guidelines utilized 
for the completion of this table are presented in Table 3.3. Each 
component was assigned a rating, and in order for these ratings to be 
comparable with the subjective evaluation performed by regional technol- 
ogists, the auditors used the APIOS Ranking Criteria (Table 3.4) as a 
guideline. A summary of the siting and operations ratings for all sites 
audited is presented in Table 3.5. 

As was described in Phase I, a marginal or poor rating 
requires that the QA Coordinator review the site and make appropriate 
changes to Improve the rating or relocate the site. A marginal rating 
is an alert that potential problems exist and a poor rating indicates 
that changes should be made immediately. 

Sites that have been classified as poor are discussed in more 
detail below, and where applicable, site changes are recommended. The 
importance of the deviations from the siting criteria should be 
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TABLE 3.2 
SuwMry ■ Phase 11 Site *udU findings 





Ear Falls 


E.L.A. 


Ouetico Centre 


1 SITING 








a) Windbreak 


180* (siiied) 


90' (burned Coniftrousl 


330* (Blxed) 


b) Ground Cover 


Poor (clay t exposed 
earth) 


Good (bedrock, aoss) 


Good 


c! Obstructions 


Yes (trees, gasoline 
storage tank) 


Ket Tower H-lOm I4in I 


None 


d) Rtp. of Topography 


Yes 


Yes 


Yes ' 


e) Roads/Transportation 


Access road t parking 

lot (gravel) 


None 


Access rd. 30*1 y (paved) 


fl Agriculture 


None 


None 


Hone 


g) Towns/Cities 


Ear Falls (<1 kn) 


None 


Hone 


hi Pollution Sources 


Open pit Iron ore 
mine 2Skm N 


None 


Hone 


11 Power Failures 


3 per year 


1-2 per year 


1-2 per year 


J) Deviations 


Poor ground cover, 
obstructions, 
proxinity to road, 
construction, open 

pit mine 


Tower - influence Is 
minimal 


Proximity to road 


k) Advantages 


Site is accessible 
and secure 


Remote and represent- 
ative 


Remote and represent- 
ative 


11 Rating 


Peer - many deviations 


Excellent 


Excellent 


11 OPERATIONS 








a) Condition of Iitstr. 

- Present 

- Past 


Chain needs tighten- 
ing, limit switch 
needs to be replaced 
1 adjusted 


Motor humming otherwise 
good 

Good 


Good 
Good 


b) Understanding 

- Has manual 

- Read manual 

- Understands Protocol 


N/A 
OPERATOR 


Sections pertaining to 

h1ffl 
Yes 
Yes 


Yes 
Yes 


c) Follows Protocol 

- Instr. Cheeks 

- Sample Handl Ing 

- Cleaning 

- Storage (sample) 

- Std. Gauge Protocol 


WAS NOT 
PRESENT 

FOR 

THE 
AUDIT 


Daily 

Good - follows protocol 

Good 

Follows protocol 


Instruaient checked 
dally - well maintained. 
Operator needs to be 
retrained in sample 
handling - possible 
source of contaul nation, 
std. gauge taken 
properly 


d) Attltudt 




Good 


Good attitude - but 
feels too auch paper 
work 


e) Reaction to Probleiw 




Good 


Good, telephones 
technologist 


f) Submission Forms 




Properly filled out 


Properly filled out 


gl Rctrtlnlng Needed 




None 


Yes - needs to be 
retrained on sample 

handling 


hi Deviations 
11 Advantage 


1 


None 

Conscientious 

Interested in network 
would like more Info on 
APIOS and coiTfilete 
operations manual 


leans over pail , same 
gloves used to pat 
down bag as to install 

- checks sampler daily 

- conscientious 


j) Rating 




Excellent 


Good (retraining) 
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TABLE 3.2 (cont'd) 
Suinnary . Phase II Site Audit Findings 





Dorset 


Whitney 


Coldwater 


I SITINS 








a1 windbreak 


360* (miied) 


360* (mixed) 


360* (deciduous) 


b\ Ground Cover 


Compound - good; 
surrounding fair 


Harginal (some exposed 
earth weeds need 
cutting) 


Good, cut grass 


c> Obstructions 


None 


None 


Hone 


4) Rep. of Topography 


Yes 


Yes 


Yes 


el Roads /Transportation 


Driveway - dirt 50m 
EKE 


None 


None 


f) Agriculture 


Hone 


None 


Yes-corn/hay (500m-lkm 

ENE) 


g) Towns/CIHes 


None 


None 


Orillia Ukm pop. 
25 .000 


n) Pollution Sources 


None 


Sawmill 3Skn SE 

(downwind) 
Marsh 3Sm ,HE 


Open pit limestone 
quarry 10- 15km NE 


1) Power failures 


2 per year 


2-3 per year 


Hon* 


J) Deviations 


Proximity to driveway 
(dirtt 

Trailers between 
parking lot and site 
to reduce influence 
of windblown contami- 
nation 

Parking lot should be 
paved 


Exposed earth proximity 
to marsh (35ml and 
sawffli 1 1 


Proximity to limestone 
quarry and plowed 
fields 


k) Advantages 


Meets an siting 
criteria 


Remote, representative, 
accessibility of power 
and operator 


Meets most siting 
criteria 


1) Rating 


Excellent 


Harginal, (good If 
ground cover improved) 


Good (*evaluate 

Influence of limestone 
quarry) 


n OPERATIONS 




1 




a) Condition of Instr. 

- Present 

- Past 


Primary good; 

secondary need 
maintenance 


Good; clutch needs 
adjustment 


Good 
Good 


b) Understanding 

- Has manual 

- Read manual 

- Understands Protocol 


Yes 
Yes 

Yes 


N/A 

OPERATOR 


Yes 

Yes 
Yes 


c) Follows Protocol 

- Instr. checks 

- Sample handling 

- Cleaning 

- Storage (sample) 

- Std. Gauge Protocol 


Yes 

Does not change 

gloves to pat down 

bag 

Yes 

Follows protocol 


WAS 

NOT 

PRESENT 

FOR 
THE 

AUDIT 


2x per week 

Does not follow protocol 

Yes - no gloves 

Porch 
Good 


dy Attitude 


Very good 




Excellent 


c) Reaction to Problems 






Good 


f) Submission Forms 


Filled out correctly 




Filled out correctly 


g) Retraining Needed 


No - just needs 
refresher 




Yes 


h) Deviations 


Pats down bag 
without changing 
gloves 




Folds bag In before It 
Is tied 


1 ) Advantages 


APIOS Regional Tech. 
- follows almost all 




Does not label bag; 
does not always use 




protocol; good 
attitude 




n«w gloves 


J) Rating 


Good 


Ci IIIC 1 


,i MaRj*Mli:(f»*«d» . tmtratft- 
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TABLE 3.2 (cont'd) 
Suimary . Phase II Site Audit Findings 



I SITING 

«) Windbreak 

bl Ground Co»er 



c) Obstructions 

d) Rep. of Topography 
•1 Roads/Transportation 

f) Agriculture 



g) Towns/Cities 

h) Pollution Sources 

1) Power Failures 

j1 Deviations 



WhifMn Creek 



kl Advantages 



1) Rating 



Z70* (deciduous) 



Sood (grass needs to 
be cut) 

None 

Yes 

Yes, County Rd. 11* 
2Sa S 

None 



None 

Sawmill 10km E 



Many - battery 
operated (solar cell 
source) 

Pro«1im1ty to road 
and saMiiill 
unreliable power 
source 

Representative, 
accessibility, good 
ground cover, no 
obstructions 

Marginal (electrical 

power) 



Port Stanley 



Merlin 



180* (deciduous) none 
in prevailing wind 
direction 

Good 



Hont 

Yes 

Access rd. (ZOra W gravel 
very light) 

Yes (tmt representative 
300n) 



Hone 



Treatment plant, oil 
heated 

3 per month 



Cultured fields Mithin 

1 km 

Proximity to roads 



Representative, no 
obstructions good ground 
cover 



Good 



180* (scattered houses, 
barns) poor 



Good 



None 

Yes 

County rd. 50m S; gravel 
access rd. 15 ra E 

Yes (but representative 
SOOni, 1km) 



Merlin <1 km 

Chatham tSkm N (40,000) 

None 



Hone, back up generator 



Proximity to vehicular, 
agricultural and human 
activity 



Representative, good 
ground cover, no 
obstruction or 
pollution sources 

Good 



11 OPERATIONS 

•) Condition of Instr. 

- Present 

- Past 



t») Understanding 

- Has manual 

- Read manual 

- Understands Protocol 

c) Follows Protocol 

- Instr. Checks 

• S«np1e Handling 

- Cleaning 

- Storage (sample) 

- Std. Gauge Protocol 

d) Attitude 

el Reaction to Problems 

f) Submission Forms 

g) Retraining Needed 
h) Deviations 

1) Advantages 

j) Rating 



Good sensor sensi- 
tivity needs adjust- 
ment 

Poor unreliable 
during winter months 



Yes 
Yes 

Yes 



Yes 

Follows protocol 

Refrigerator 
Follows protocol 

Excellent 

Good - Informs tech. 
of any problems 

Completed thoroughly 

Ho 
None 



Conscientious, good 
attitude 

Excellent 



Good sensor sensitivity 
needs adjustment 

Good - but some problems 
with circuit breaker 
tripping 



Yes 
Yes 
Yes 



Dally 

Uses only 1 pr.of gloves 
Cleans hood gasket 
Cabinet In laboratory 
Follows protocol 

Excellent 

Good - Informs tech. of 
any problems 

Completed thoroughly 

Yes 

Does not wear new gloves 
to touch Inside of bag 

Good attitude 



Good 



Good sensor sensitivity 
needs adjustment 

Good 



Yes 
Yes 
Yes 



1 per week 

Follows protocol 

Yes 

Box In pump house 

Follows protocol 

Excellent 

Good 

Completed thoroughly 

No 

None 

Good attitude follows 
all protocol 



Excellent 
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TABLE 3.2 (cont'd) 













Forbes Twp 


Pickle Lake 


Winlsk 


I SITING 








t) Ulndbretk 


360' (alxcd) 


300* (Coniferous) 


270* (Coniferous) 


b) Ground Cover 


Good 


Good 


Good 


e) Obstructions 


Hone 


Trees (E. SE) Towtr (HE) 


None 


d) Rtp. of Topography 


res 


Yes 


res 


t) Ro«<l»AriBtport«tton 


None 


Hwy. 646, 46m I (very 
light) 


Snowmobile road In, 
Airport Hanger (400m) 


f) Agriculture 


Mon« 


Monc 


None 


gj Towns/CUlei 


None 


P1ck1« Lake (<1 km) 


None 


hi Pollution Sources 


Power plant (35 km) 
(downwind) 


Mood burning source of 
heat 


Diesel generator 
(24 hrs/day) 


1) Power Fjilures 


3 per year 


None 


None (back-up generator) 


J) Deviations 


Proximity to power 
plant 


Obstructions, wood 
burning source of heat 


Proximity to snowmobile 
trail, airport hanger i 
runway, diesel generator 


k) Advantages 


Remote, representative 


Remote, representative 


Remote, representalve 


1) Rating 


Good 


Good - trees should be 

removed 


Marginal 


II OPERATIONS 








i) CondUlon of Instr. 
- Present 


good 


Clutch too tight >3S lbs 
sensor grid blackened 


Not operating at time of 
audit 


• Past 


Ho problens 




No maintenance performed 


b) understanding 

- Has Mnual 
• Read manual 

- Understands Protocol 


Yes {event section) 

res 

res 


res (cumulative section) 

res 

res 


Yes 1 

No 
NO 


e) Follows Protocol 

- Instr. Checks 

- Sample Handl Ing 

- Cleaning 

- Storage (sample) 

- Std Gauge Protocol 


res 

Follows protocol 
Weekly 

Refrigerator 
Follows protocol 


res - dally 
Follows protocol 
Cleans gasket but no 
gloves 

Refrlg. until shipped 
Follows protocol 


None 

Does not follow protocol 

None 

Kept In repair shop 
Std. gauge empty at 
time of audit 


i) Attitude 


Excellent 


Poor - teaporary oper. 


Very poor 


•) Reaction to Problems 


Good - fretjuent 
contact with 
technician 


Good 


Does not call when 
instruments down 


f) Sub«1ss1on Foms 


Completed thoroughly 


Completed thoroughly 


Hot filled out properly 


g) Retraining HMded 


No 


NO 


res 


h) Deviations 


Hone 


Hone 


Numerous - too many to 
list 


1) Advantages 


Conscientious 1 
follows protocol 


Follows protocol 


None 


J) Rating 


Excellent 


Good 


Poor 
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TABLE 3.2 (cont'd) 
Summary - Phase II Site Audit Findings 





Gowganda 






I SITING 








8) Windbreak 


165* (mixed) 






b) Ground Cover 


Good (weeds to be cut) 






c) Obstructions 


Tower (H-12m 20m S£) 






d) Rep. of Topography 


Yes 






e) Roads/Transportation 


Access Rd. gravel 25iii 
S i E 






f) Agriculture 


None 






g) Towns/Cities 


Gowganda < 1 km 






h) Pollution Sources 


None 






1) Power Failures 


Seldom 






j) Deviations 
k) Advantages 


Poor windbreak 

especially on Lake 

side 

vandalism 

Remote, representative 






1) Rating 


Good (improve 
windbreak on lake side 
cut grass 2x year) 






II OPERATIONS 








<) Condition of Instr. 

- Present 

- Past 


Good 
Good 






b) Understanding 

- Has manual 

- Read manual 

- Understands Protocol 


Yes 
Yes 
Yes 






c) Follows Protocol 

- Instr. Checks 

- Sample Handl ing 

- Cleaning 

- Storage (sample) 

- Std Gauge Protocol 


Yes 

Follows protocol 

Does not clean sensor 

grids 

In insulated box 

Follows protocol 






d) Attitude 


Very good 






«) Reaction to Problems 








f) Submission Forms 


Filled out completely 






g) Retraining Needed 


Operator unsure of 
sampling procedures 
should be retrained 






h) Deviations 

j 








1) Advantages 








i) Rating 


Excellent 
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TABLE 3.3 
SuldeHnes for the Cowplttlon of ■Siwary - »>h«*« H Sitt todlt Findings" 



Coi^onent 



1 SITIWG 

i) Ulndbrtak 

b) firound Cover 

e) Obstructions 

d) Rep. of Topogrtphy 

e) Ro«dsArin»port*t1on 

f) Agriculture 

g) Toxns/cnies 
h) Pollution Sources 

1) Power Failures 

Jl Deviations 

Icy Advantages 

1) Rating 



Suldtllnes 



At how Mny degrees of the coa^iss Is there a vlndbreak and describe type? 

Sood ground cover Is cut grass or boss; poor ground cover Is exposed 
soil. 

Uo object should be slthln 2.5 tiBes object height to saapler. 

The topography and vegetative ground cover should be representative of 
surrounding area (50 ka). 

Mo Mjor transportation line (road/railway) wtthin one k11o«ter of site 
ts It nay be a contamination source. 

No agriculture which aay provide a conUnlnatlon source within one 
kilometre of the site. 

No urbanized areas close to the sampling site (distance depends on 
sire of town/city). 

Mo ulnor pollution sources within 1 km (e.g. sewage lagoons, salt and 
sand piles, gravel pits). No najor pollution sources within 50 ki» 
(power plants, suiters, open pit Bines, pulp and paper plants, etc.). 

How Bany power failures tn there per year? 

What are the deviations of the site froo the APIOS siting criteria? 

Mhat are the advantages of the site? 

Objective evaluation based on finding fro* item (a) to (k) above. 



11 OPERATIONS 

a) Conditions of Inttr, 

- Present 

- Past 

b) Understanding 

- Has Manual 

- Read Manual 

- Understands 
Protocol 

c) Follows Protocol 

- Instr. Checks 

- Sample Handling 

- Cleaning 

- Storage (sample) 

- Std. Gauge Protocol 

d) Attitude 



e) Reaction to Probleas 

fl SubBlsslon Foras 

g) Retraining Needed 

h) Deviations 

1 ) Advantage 

j) Rating 



Findings of tests and observations carried out during the site audit 
to deternlne the past and present performance of the instruments. 
The Information Is based on calibration and configuration checks, 
operator coaments and auditor's observations. 

Understanding by the operator of procedures. Has he read the APIOS 
■anual, does he undersUnd It. does he undersUnd the sampling 
protocol? 



Does the operator check the performance of the Instruments, handle 
the sample as per protocol, clean the required components properly 
(knife edge, gasket), store sanple In a location that would prevent 
sample degradation and follow only updated sailing protocols? 



Operator's attitude towards the procedures that he/she must follow and 
towards the APIOS prograa in general. 

Does the operator get In touch with the regional technologist promptly If 
a problem is noted? 

Are the submission forms filled out properly and coipletely? 

Is the operator following Bost current procedures? Is retraining 
required? 

What procedures performed by the operator deviate fro* current APIOS 
operator sampling protocol? 

Site operations advantages (e.g. attitude, protocol follwed). 

Objective evaluation based on the findings froB (a) to (1) above. 
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TABLE 3.4 
APIOS Ranking Criteria 



Rating 


Site 


Operator 


Excellent 

I 


An excellent site should meet all siting 
criteria. It is easily accessible 
year-round and has no record of any 
vandalism. 


An excellent operator is one who 
follows APIOS handling procedures at 
all times. Samples are collected as 
scheduled, routine instrument checks 
are made, and detailed log books are 
maintained. The operator maintains 
an interest in the program and 
regards his/her role as an Important 
function. The excellent operator is 
one who goes out of his way to ensure 
the collection of quality samples. 
These operators frequently 
participate in special studies. 


Good 

1 


These sites meet the majority of siting 
criteria. Any deviations from siting 
criteria do not significantly affect the 
integrity of samples collected. Examples 
of this category might include sites that 
do not have windbreaks on all compass 
points; sites located near agricultural 
areas; sites that have accessibility 
problems during certain times; or sites 
that may have objects <2.5 times the 
height away from the sampler {but not <1 
time the height away). 


A good operator follows APIOS proce- 
dures. Samples are collected, rou- 
tine instrument checks are made, and 
a log book is maintained. The opera- 
tor may not be sincerely Interested 
in the program but procedures are 
followed as required. 


Marginal 


These sites need improvement in order to 
be considered a good site. Examples 
might be sites with marginal ground 
cover, or sites with object(s) near the 
sampler (1 time the height) that may 
affect the sampler occasionally. Data 
from these sites should be closely 
scrutinized. 


Marginal operators do not always 
follow standardized procedures. They 
occasionally forget to change samples 
and keep sketchy log books. If any. 
The low interest of these operators, 
combined with their performance, do 
not allow the collection of quality 
data on a regular basis. 


Poor 


These sites should either be removed or 
significantly improved. They do not meet 
important siting criteria and samples are 
routinely contaminated. Data collected 
is unreliable. 


Operators In this category do not 
follow procedures and maintain no 
interest in the program. If 
performance cannot be upgraded, then 
an alternative operator should be 
Immediately located. If no other 
operator can be found, consideration 
should be given to removing the 
site. 
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Siting 



Operations 



Excellent 



Good 

Marginal 

Poor 

Unrated 
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TABLE 3.5 
Sunwiary of Site Audit Ratings 



Excellent 



Good 



E.L.A. 



Dorset^ 
Quetico 
Centre^ 



Gowganda 
Merlin 
Forbes Twp 



Port Stanley 
Pickle Lake' 



Col dwater"* 



Marginal 



Whitman Crk.^ 



Poor 



Winisk^ 
Whitney^ 



Ear 

Falls^ 



1. Unreliable power source, numerous events missed. , ,j 

2. To reduce influence of windblown contamination, parking lot should 

be paved. 

3. Good if ground cover improved. 

4. Evaluate influence of limestone quarry; retrain operators 
Immediately. 

5. Many deviations, site to be reviewed by QA Coordinator. 

6. Operator needs upgrading. 

7. Obstructions (trees) should be removed. New operator to be 
trai ned. 

8. Poor operator; should be replaced. 
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evaluated. Action to correct the faults and/or qualify the data collec- 
ted at these sites should be taken. 

1. Ear Falls 

The cumulative monitoring site at Ear Falls was rated as 
"poor" due to numerous deviations from the APIOS siting criteria. These 
deviations include: 

poor ground cover (gravel & dirt) 

on-site obstructions include two coniferous trees 10m NE and a 

gasoline holding tank 8ra ME 

proximity to a small industrial park (3 km W) 

frequency of vehicular traffic near the gasoline holding tank 

location of site (adjacent to a utility garage and storage 

area) 

It is strongly recommended that the data collected to date be 
carefully evaluated and this site be relocated in order to collect more 
representative samples. 



Concord Siirntiiit I'oriioMlion 



3.13 - 



2. Winisk 

The operations component of the Winisk site was rated as poor. 
The Sangamo collector was in poor condition at the time of the audit. 
The GFIC (ground fault Interrupter circuit) had tripped, the fuse was 
blown, one of the sensor grids was broken, the clutch was set too high 
and the old knife edge collar (with a gap) had not been replaced. 

Historical data from this site should be carefully examined. 
The auditor suspects that bulk samples were being collected at this site 
during the last few months (if not longer). Samples are often collected 
weeks and even months late. The operator lacks interest and has a poor 
attitude towards the APIOS programme. As well, he does not follow any 
of the documented sample handling procedures. 

Deviations from the siting criteria include the proximity of 
the site to the snowmobile road and the airport. However, given the 
remoteness and operator accessibility, it is a reasonable compromise. 
The site could be Improved by limiting the snowmobile traffic. 

Operator and communication problems at this site are extreme 
and lead to an overall unacceptable rating. The auditor believes that 
it would be very difficult to train this operator due to attitude. 
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comnwnl cation and other problems. It is recommended that a new operator 
be found and thoroughly trained, otherwise the site should be removed. 

Some general observations made during the audit are discussed 
below: 

1. The majority of the operators audited have shown marked 
improvements in the performance of the sample handling proce- 
dures outlined in the APIOS Technical and Operating Manual 
(Bardswick, 1983). Most technologists have either retrained, 
upgraded and/or reviewed the current operating procedures with 
their operators. At some sites, the "procedural short list" 
from the manual has been posted near the instruments. This 
allows the operator to quickly check that he is following the 
procedure correctly. Posting procedural short lists should be 
considered at all sites. 

2. The following inconsistencies were noted in sampler configura- 
tion. 

a) The southwestern and central regions had no aluminum 
foil wrapped around the collection vessels to reduce 
sample Toss due to evaporation. 
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b) Aerochem Metrics dry buckets were still In place in the 
northwestern region. This was the only region utilizing 
these buckets. 

Uniformity of instrument configuration throughout the network 
should be instituted. Inconsistencies in configuration from 
region to region should be assessed by the QA Coordinator for 
their effect on sample quality. 

3. Table 3.6 summarizes the instrument checks performed during 
the instrument audits. In general, most instruments were 
operating within the specifications set out by APIOS. How- 
ever, the co-located Sangamo Type A precipitation collector in 
Dorset requires immediate attention. 

4. To improve the seal between the collector hood and the bucket 
of the cumulative precipitation collectors, new gaskets have 
been attached to the underside of the Sangamo/M.I .C. hoods at 
all cumulative sites (except Dorset because of an on-going 
precision study). These new gaskets provide a tight seal and 
prevent both sample loss due to evaporation and contaminants 
from entering the sample collection containers. 

5. Revised site documentation for the central region has not yet 
been completed. This should be carried out as soon as 
possible. 

Qjtxonl Suonlili( Cunxir.itinn 



D 

5: 



TABLE 3.6 
Instruaent PerfonMnce Audtt iimmtry 



Region: Centril 





NIC 


Orifice 


Collec- 
tor 




Hood 


Instruaent 
Oel«y TIae 


Instruaent 


Sensor 
Grid 


Clutch 
Adjust- 


Halt Switch 






Storage Gauge 




Orifice 




Site 


Scrlil f 


Height 


Are* 


Level 


Movenent 


(see* 


Sensitivity 


Condition 


aent 


Condition 


G«sket Seal 


Caa«nt 


Height 


Size 


Level 


Wittney 

! 


HIC 
Type A 


2 • 


Round 

M CB 


Yes 


-Noveaent 

saooth 
-Hood docs 

drop on 

knife edge 


100 


200 KQ 


Good 


>3S lbs. 


-Meeds 

idjustaent 
-Too auch 

pressure on 

support 

br«cket 


Good 

-New gasket 
InsUlled 


Tech Installed 
aesh screen on 
storage gauge 
to keep debris 
t birds out 


1.32 a 


Round 
28 ca 


Yes 


Dorset 


PrlBiry 
MIC 
Type A 


1.95 ■ 


Round 

20 ca 


Yet 


Good 


79 


200 KQ 


Good 


>35 lbs. 


Good 


Good 

-Old gasket 
used for 

special 
studies 




1.26 a 


Out of 
round 


No 




Secondary 


1.9S ■ 


Round 


res 


-Mot saooth 


86 


3i* KQ 


Good 


>3S lbs. 


-Needs 


Good 


Sangaao 


1.24 a 


Out of 


No 




HIC 




20 ca 




-Drops 










(djustaent 




* Off switch 




round 






T/pe A 








suddenly 










-\wf)ropeT 
contact 




not working 

• Motor runs 
on after 
































titstruaent 










. 






















Is turned 




















■ 












off 








Cold- 


MIC 


2 m 


Round 


Yes 


Good 


B6 


260 Kfi 


Good 


14 lbs. 


Good 


Good 




92 ca 


Out of 


Yes 


M«ttr 


Type A 


20 CB 
















-New gasket 






round 





I 






TABLE 3.6 (cont'dj 
Instruaent Perfonwnce Audit Summry 



Region: Northwest/Northeast 



G 

3- 



a. 






site 


MIC 
Serial f 


Orifice 
Height 


Collec- 
tor 
Area 


Level 


Hood 
Hovenent 


Instruaent 

Delay Time 

(seel 


Instruaent 
Sensitivity 


Sensor 

Grid 

Condition 


Clutch 

Adjust- 

■ent 


Li alt Switch 
Condition 


Gasket Seal 


CoaM-nt 


Storage Gauge 


Height 


Orifice 
Size 


Level 


E.L.A. 


MIC 
type A 


2 m 


Round 
20 cm 


Ho 


Good 


80 


200 Kfl 


Good 


>30 lbs. 


Good 


Good 
-New gasket 


*Fall wrapped 
around long 
tube 

*Motor huanlng 


1.40 a 


Round 
28 ca 


Tes 


Eir 
Fills 


HIC 
Type A 


2 > 


Round 
20 en 


Yes 


-Hood drops 
on both wet 
1 dry side 


80 


M/A 


Good 


20 lbs. 


-Heeds adjust- 
ment 

-Too auch pres- 
sure on 1 lal t 
switch bracket 


Good 
-New gasket 


*fo11 wrapped 
around long 
tube 


1.50 a 


Round 


tes 


Quetico 

Centre 


HIC 
Type A 


Z ■ 


Round 
20 cm 


Yes 


Good 


87 


200 Kfl 


-Hot to 
touch 
-One grid 
blickened 


20 lbs. 


Good 


Good 
-New Gasket 


'Foil wrapped 
•round long 
tube 








Pickle 
Like 


HIC 
Type A 


2 m 


Round 
20 cm 


tes 


Good 


110 


N/A 


-One grid 
blackened 


>35 lbs. 


Good 


Good 
-Hew gasket 


•foil wrapped 
around long 
tube 


1.30 a 


Round 
30 ca 


Yes 


Utnlsk 


HIC 
Type A 


2 ■ 


Round 
19 ca 


No 


Good 


30 


MO Kn 


-One grid 
broken 


>3S lbs. 


Good 


Good 

-Old knife 
edge 
collar 


•GFIC tripped 
t fuse blown 

-One socket 
exposed to 
aolsture 








GoHganda 


HIC 
Type A 


2 ■ 


Round 
19.5 c« 


Tes 


Good 


35 


23 KQ 


Good 
very vara 


>3S lbs. 


Good 


Good 
-Hew gasket 




t.30 a 


Round 
29. Sea 


Yes 



I 

■ 

-J 
I 



TABLE 3.6 (cont'd) 
Instruaent Perforwance Audit Suwwry 



Region: SouthMest 





MIC 


Orifice 


Collec- 
tor 




Hood 


Instnnent 
Delay TiBe 


InstniBent 


Sensor 
Grid 


Clutch 
Adjust- 


LiBit SmUcH 






Storage Gauge 




Orifice 




Site 


Ser1*1 1 


Height 


Arei 


Level 


Hovenent 


(sec) 


Sensitivity 


Condition 


aent 


Condition 


Gasket Seal 


CoMent 


Height 


Size 


Level 


Herlln 


NIC 
Type A 


1.99 ■ 


Round 

20 CM 


Yes 


Good 


106 


100 li 


Good 


IS lbs. 


Good 


Good 
-New gasket 




1.48 ■ 


Round 

20 CB 


Tes 


Port 


PrlMry 


1.93 B 


Round 


Tes 


Good 


95 


40 a 


Good 


16 lbs. 


Good 


Good 




1.61 ■ 


Hot 


Yes 


StMley 


MIC 

Type A 




20 CB 
















-Mew gasket 






round 
29.2 CB 
28.8 CB 






Secondary 


1.9fi ■ 


Round 


Yes 


-Not SBOOth 


90 


100 a 


Good 


16 lbs. 


Good 


Good 


*GFIC trips 


1.73 B 


Round 


Yes 




MIC 




20 CB 




-Jerks 












-Hew gasket 


frequently 




28.8 CB 






Type A 






















up to 3 x 
per Bonth 









'Mi 

CO 



Q 
t 



1^ 



TABLE 3.6 (cont'd) 
Instrument Performance Audit Sunwary 





Aerochem Metric ! 


Rain Gauge 


Site 


Sampler 
Orifice 


Level 


Comments 


Gasket 
Seal 


Gasket 
Condition 


Sensor 
Condition 


Orifice 
Height 


Level 


Clean 


Forbes Twp. 
Primary 


Round 
28.5 cm 


Yes 


- On/off switch 
Installed 

- Dry bucket still In 
place i 


Good 


Good 


Good 

1 


43 cm 


Yes 


Yes 


Forbes Twp. 
Secondary 


Round 
28.5 cm 


Yes 


- On/off switch 
Installed 

- Dry bucket still In 
place 


Good 

t 


Good 


Good 


44 cm 


Yes 


Yes 


Whitman 
Creek 


Round 
28.5 cm 


Yes 


- Power problems 

- Many events missed 


Good 


Good 


Good 


76.5 cm 


Yes 


Yes 


Quetico 
Centre 


Round 
28.5 cm 


Yes 


- Dry bucket In place 

- Water dripping from 
sensor head 


Poor 


Moisture 
on Inside 
of gasket 


Good 


42.5 cm 


Yes 


Yes 



I 

I 
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3.2 Technologist Performance 

The APIOS technologists carry a major portion of the responsi- 
bility for ensuring that the network operating procedures are correctly 
carried out. They must ensure that the instruments are working properly 
and that all relevant data are recorded and reported. 

During the Phase II Audit, two of the regional technologists 
were visited, one in the northwestern region and one in the central 
region. The findings of the technologists' audit indicate that the 
regional technologists were following operating and technical procedures 
as described in the APIOS Technical and Operating Manual (Bardswick, 
1983). 

The technologists maintain close conwiunication with the opera- 
tors in their region through frequent telephone contact and site visits. 
Event sites are visited once a week, and cumulative sites at least once 
a month in the central region, and once every two months in the 
northwest region. The technologists, where possible, meet with the 
operators during these visits and discuss sampling and instrument prob- 
lems. They also provide their operators with literature on "Acid Rain", 
such as MOE publications, articles from periodicals and newspapers, and 
will answer any questions the operators may have. The operators find 
these publications and discussions informative and technologists should 
be encouraged 
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to participate in this type of information exchange. It is recommended 
that the technologists continue to maintain close comnuni cation with 
operators, encourage operators to contact them for any reason, and try 
to co-ordinate their site visits so that they meet with the operators as 
often as possible. 

Technologists have either reviewed operating procedures or 
retrained their operators within the last four months. Although 
operators were, for the most part, following standard operating proce- 
dures, It is recommended that the technologist carry out a procedures 
audit at least twice a year to ensure that operators are following the 
current protocols. Technologists should ensure that the operators are 
not contaminating the sample while performing these procedures. 

In the northwestern region where sites are widespread, and the 
visit frequency is low (every few months) due to travel restraints, the 
regional technologist requires duplicates of operator sheets, monthly 
site summaries (see Figure 3.1) and log books. One copy remains at the 
site and the second copy is sent to the technologist. The purpose of 
this is two-fold: 

a) It allows the technologist to get an overall view of what is 
happening in the region, detect any problems, and schedule 
site visits. 
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FIGURE 3.1 
Operator MontMy Site Sumwry Shee^t 



C 



Collection Period 
Operators Cumulative Log Sheet Fro«: to: Site Ma»e: 


Instrument 


Date 


Tine 


Operation 

Check 


Storage Gauge 


Volume 
(L) 


Time 
iHrs) 


Rotameter 
(1pm) 


Comnents-'Observatl ons 


Depth 


Temp. 
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b) It provides a QC check to ensure that the operator sheets are 
filled out correctly and completely and that all relevant 
sample information is documented, e.g. identifying potential 
sources of sample contamination, instrument malfunctions, 
etc. 

It is recommmended that this system be adopted by the network where 
sites are infrequently visited by technologists due to time and travel 
restraints. 

The northwestern regional technologist has also developed 
Quality Assurance Activity Log Sheets (see Figures 3.2, 3.3) for the 
cumulative and event QA activities carried out in the region. This 
system enables the technologist to plan QA/QC activities necessary to 
meet the network requirements and at the same time monitor the 
effectiveness of the QA/QC system in his region. This system should 
also be considered for adoption by the network. 

The northwestern Regional Office provides the regional techno- 
logist with a student for two to four days a week. The majority of the 
student's time is spent on sample processing (precipitation samples and 
filters). Initially, the technologist should check all of the work 
carried out by the student (e.g. checks field submission sheets, filter 
loading and unloading procedures, etc.) to ensure that: a) the student 
has been trained adequately and b) the quality of the work is accept- 
able. 

Com. nrd ^c iontiiic ». i irixjMiu mi 



- 3.24 



FIGURE 3.2 
QA Activity log Sheet ^ Event Sampling 



Region 



Quality Assurance Activity log - Event Sampling 
Year Technician 



Activity 



Sent out or 
Requested Done 



June 



Blank 
Bags 



Sept. 



Dec. 



Mar. 



June 



Handling 

Filter 

Bags 



Sept. 



Dec. 



Mar. 



Site 



operator Audit 



Mar. 



Field 
Blanks 
and 
Duplicates 



Apr. 



May 



June 



July 



Aug. 



Sept. 



Oct. 



Nov. 



Dec. 



Jan. 



Feb. 



Composites 



Feb. 



May 



Aug. 



Nov. 



Forbes 
6081 



Otter 

Island 

6111 



Ouetico 

Centre 

6071 



Fernberg 
6051 
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FIGURE 3.3 
QA Activity log Sheet - Cumulative SawpTIng 



Qua! 11 
Region 


ty Assurance Acti 
rear 


vlty Log - Cumulative Sampll 
Technician 


^ 














Sent out or 
Requested 


ACTIVITY 


Blank 
Bags 


Handling 
Filter 
Blanks 


Volume 

Heasurement 

Checks 


Composites 


eoH 


6121 


6031 


6041 


6051 


6071 


6091 


6111 


6061 


Jun. 


























Sep. 




1 






















Dec. 




li 1 






















Har. 


























Operator 
Audit 
1984 




j 























I 

to 



cn 






3.26 - 



The technologist performs scheduled maintenance, repairs, 
calibrations, QA/QC activities, data screening and validation, upgrading 
of sites and arranging operator and site visits. The technologist 
feels, however, that the "hands on" training he received on some 
instrumentation was inadequate (specifically, the sequential sampler and 
modified Aerochem Metric sampler). This technologist should be 
retrained on some instrumentation and the auditor suggests that this 
retraining be carried out in conjunction with the quarterly APIOS 
Technologist meeting. 

The APIOS Technical and Operating Manual (Bardswick, 1983) 
lists spare instrument parts that the technologists should have on hand 
to be able to quickly maintain their instruments. This list is based on 
maintenance requirements for five precipitation collectors: 

1 precipitation sampler 1 motor 

2 printed circuit boards 10 limit switches 
4 sensor grids 1 gasket 

2 capacitors 

The technologists were found not to comply with this 
guideline. The technologists either did not order sufficient supplies 
on their 6 month request forms, or when they did run out of a particular 
Item they waited until the last moment to notify the Head Office 
Technologist. As a result, instrument downtime is longer than necessary 
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due to the length of time required for delivery of parts from Head 
Office; often two weeks are required to receive the necessary parts for 
repair of samplers. In order to maximize data completeness by reducing 
downtime, technologists should ensure that they have on hand at least 
the recommended supply of spare parts as a minimum at all times. 

The technologist in the central region is responsible for the 
operation and maintenance of a special studies site in Dorset. This 
involves operating and maintaining many co-located instruments such as 
sequential samplers, low volume air samplers, daily and cumulative 
precipitation samplers, gauges, continuous monitors, as well as conduc- 
ting various on-going special studies. 

The majority of his time is spent collecting, processing and 
documenting APIOS routine and special studies samples. With the addi- 
tional responsibility of installing, operating, maintaining and co- 
ordinating long and short-term special studies, the technologist is 
often unable to perform regularly scheduled preventive maintenance 
procedures, and as a result some of the instruments in this region have 
not had preventive maintenance carried out. It is essential that all 
Instruments be serviced and calibrated on a routine basis to ensure 
reliability and accuracy. The Instruments in this region should be 
serviced and calibrated when required. If the technologist is unable to 
carry out this task then manpower should be made available to do so. 
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The technologists visited during the audit (Phase I and Phase 
II) have proven to be a real asset to the APIOS programme. Because of 
the unpredictable nature of field work, these technologists often make 
valuable subjective decisions regarding the operations of instruments, 
performance of operators, procedures for data handling and relevant data 
recording. They were found to be conscientious and concerned about 
their duties and generally their performances have ensured the genera- 
tion of good quality data. 

3.3 Performance Audit of Data Handling and Validation Procedures - 
Daily Air Concentration Data 

Data validation and screening systems are used to evaluate the 
raw data before Including them in the final database. A number of 
screening techniques, checks and statistical routines are run on the 
data to flag unusual or suspect data. A careful scrutiny of these 
suspect data is performed before the decision to include these data in 
the database is made. Specific data validation procedures have been 
developed and documented for daily ambient air data and are described 
elsewhere (Kirk, 1984). Basically, the validation procedures utilize 
two approaches. The first approach Is to apply gross limit checks to 
identify unlikely values and the second approach is to examine the 
sample structure by principal component analysis. 
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The performance audit checked whether the documented screening 
and validation checks were being properly carried out. Errors in the 
data handling system were detected with the use of non-routine manual 
screening techniques. 

The audited data set consisted of data collected at Fernberg 
and Longwoods during the months of February, March, June and July, 
1982. 

A description of the checks performed, along with the 
findings, are described below. 

1. Check - Field sheets were screened manually to ensure that 
they had been filled out completely and correctly by the regional tech- 
nologist. The field sheets contain information about the samples sub- 
mitted, such as the site name and number, sampling period, filter type, 
gas meter reading, sample volume, rotameter reading, sampling port 
number, sampling start and end dates, associated passive sample numbers 
and field comments about the sample. A blank air sampling field data 
sheet is shown in Figure 3.4. 

Findings - Fernberg 

• All field sheets for the months of February and March were 
missing associated passive sample numbers and sample type 
(e.g. active, passive or blank). 
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FIGURE 3.4 
APIOS Air Sampling Field Data Sheet 
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Filter lot numbers for Teflon, Nylon and Whatman 41 filters 
were missing for the period February 1-10, 1982 and the filter 
lot number for Teflon filters was recorded as a "question 
mark" for the period June 22 - July 6, 1982. 

Associated passive sample numbers were recorded incorrectly on 
two occasions: 

Reported Associated Actual Associated 
Sample Number Passive Number(s) Passive Number(s) 

95441 95432/95448 95440/95448 

95443 95442 95448 

All field sheets for the study period were missing initial and 
final rotameter readings and operation check. 

Some switched filter dates for "set on tower" are recorded 
incorrectly. The switched filters are left on the tower for a 
two week period, the first week in port #8 and the second week 
in port #1. On some field sheets the date "set on the tower" 
was recorded as the first day of the second week (however, 
this date is not entered into the database). 

All sample dates, times, station identifiers and sampling port 
numbers were recorded correctly. 
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Longwoods 

o Sampling end date for sample number 15347 was recorded In- 
correctly as 15/2/82. This was a two day sample beginning on 
14/2/82 and ending on 16/2/82 (sample volume = 52680 1). 

» All station identifiers, sampling port numbers and associated 
passive numbers had been recorded correctly. 

• Some data recorded as "date set on tower" were Incorrect for 
switched filters (see Fernberg). 

Manual screening of field data sheets prior to submission 
would have prevented omissions and errors. It Is recommended that the 
technologists visually check the field sheets for missing information, 
Incorrect dates, Incorrect times and illogical coding prior to submis- 
sion. These manual screening techniques are described in the APIOS 
Technical and Operating Manual along with procedures to be followed when 
filling out these data sheets. When an omission or error is found, a 
revised or corrected field sheet should be submitted. 
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2. Check - Field sheets were checked against 1982 Dally Ambient 
Air Concentration Listings (MOE, 1984). This Involved comparing the 
data recorded on the field sheet to the APIOS published data and check- 
ing for transcription errors and omissions. 

Findings - Transcription errors were not found. In all cases, 
the information listed in the database complemented the field sheets. 
Only one Instance of a data point omission was found in the published 
report. A field sheet and LIS output for a sample collected at Fernberg 
(February 9-10, 1982; #30139) was found but the corresponding concentra- 
tion listings were not reported. 

3. Check - Summary of field and office comments. This involved 
tabulating the office and field comments for all daily air concentration 
measurement sites for the period January 1 to December 31, 1982. This 
table provides a quick overview of the types of comments which are being 
appended to analysed samples. Frequently, recurring field or office 
comments can sometimes be good indicators of sample contamination, poor 
instrumentation or non representative siting. 

Finding - Table 3.7 summarizes the frequency of comment codes. 
Dorset had the greatest number of sampler malfunctions 35 of 334 samples 
(10%) and abnormal sampling periods 16 of 334 samples (5%). However, 
the majority of these malfunctions did not affect the samples. Since 
the Dorset technologist Is on-site, any sequencing problems were always 
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TABLE 3.7 



Frequency of Field and Office Coipents - Dally Dry Data 



T5B? 



Site 


No. 

of 

Records 


% 
Data 
Complete- 
ness 
































Lab Comnent 


t-ieid toMMenis 


Comnents 


Result Remark Code 


A 


B 


C 


D 


E 


F 


G 


H 


1 


J 


K 


F 


P 


Z 


X 


* 


> 


< T 


< W 


A 


U 


P 


< P 


Longwoods 


343 


94 


3 






I 






2 


3 


9 


1 


2 


2 




6 


3 


86 


71 


4 


11 




18 






Dorset 


334 


91 


35 








2 




S 


I 


6 


35 








16 




21 


16 


58 


122 




8 






Charleston 
Lake 


340 


93 


16 


1 




6 






7 








I 


2 




8 




36 


51 


21 


47 




23 






Fernberg 


317 


87 


5 


B 


1 




3 








2 






5 




3 




28 


8 


52 


151 




27 








3 



Fleld Comnient Code Index 

A - Sampler Malfunction 

B - Known Hydro Failure 

C - Suspected Hydro Failure 

- Known Filter Pack Leak 

E - Suspected Filter Pack Leak 

F - Gas Meter not equal to Rotameter 

Office Comment Code Index 



G - FlOM Line Problems 
H - Known Contamination 
I - Suspected Contamination 
J - Heavy Dew or Fog 
K - Sample not Submitted 






F - Data Invalidated - Flow volume rate less than 10,000 1 per day 
P - Passive missing - Average passive results used as blank correction 
Z - Abnormal sampling period 
X - Sample lost 

Result Remark Code Index 

> - Actual result greater than value reported 

< - Actual result less than value reported 
<T - Actual result less than criterion of detection 
<H - No response, minimum possible result reported 

A - Approximate value 

U - Unreliable result 

P - Not corrected for passive 

<P - Not corrected for passive - Reported value is a detection limit 
* - No data reported 
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fixed before a new pack began sampling. Charleston Lake had the second 
highest record of these occurrences, 5% (sample malfunctions) and 25% 
(abnormal sampling periods). Fernberg reported the largest number of 
invalid data due to insufficient sample volume (<1000 litres per day) 
2%. 

To determine the frequency of occurrences per site due to 
hydro failure, contamination or flow volume problems, field comment 
codes associated with each of these problems were combined and a 
percentage calculated. The results are tabulated in Table 3.8. 

4. Check - Data Validation Procedures. This involved determining 
if the procedures for validating daily dry data described in Kirk (1984) 
have been correctly executed. These procedures involve qualifying 
sample records by three methods. 

1. Field comments - which are appended to the sample records at 
the point of collection to reflect sample quality and possible 
sources of contamination. 

2. Office comments - which are appended to sample records to 
indicate failure of specific integrity tests or sample 
representativeness. 

3. Validation flags - which are appended to individual analytical 
results if outliers are detected or gross limits fail. 
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TABLE 3.8 

Frequency of Field Coniments Per Site for Hydro Failure. Flow Volume 

Problems and Contamination 
Event Air Sampling Network (1982) 



% 



Site 




Hydro Fail 
B+G 


ure 


Fl 


ow Volume 
D+E+F+G 

<1 


Contamination 
H+I 


Longwoods 







3 


Dorset 











<2 


Z 


Charleston 


Lake 


<I 






4 





Fernberg 




<2 






<1 


<1 



Contofcl >K k'liiiiK (i >i|X)i.ttii)n 



- 3.37 



Field comment codes appended to samples from Longwoods and 
Fernberg during the study period (e.g. February. March, June and July, 
1982) were Investigated to determine if the appropriate office comments 
and validation flags had been applied. Field data sheets, LIS outputs 
and reported data concentrations were compared. Samples which exceeded 
the gross limits were tabulated and the data examined. 

Findings - The procedures for qualifying data were for the 
most part correctly executed. Some modifications may be required when 
reporting data with the following field codes: I (suspected contamina- 
tion) and H (known contamination). When a single filter (Nylon, Teflon 
or Whatman 41) is potentially contaminated while being unloaded due to 
improper filter handling techniques (e.g. dropped on floor, touched by 
technologists bare hand, etc.) a "result remark code" should be appended 
to the analytical result associated with the analysis of that filter 
rather than to the entire sample, i.e. for nylon filter a remark result 
code should be appended to the N-HNO3 result. This also applies to a 
single filter with the code H (know contamination) appended to it. 
Table 3.9 summarizes the field comments investigated. 

Tables 3-10 and 3.11 summarize samples in which a parameter 
exceeded the upper or lower gross limit. The following observations 
were made: 



Concord .VnnlitK CiMpH.ilion 



- 3.38 - 

TABLE 3.9 
Pit* Quallflcetlon Check 



Site 


Field 
Hunter 


Field 
CoBwent 


Office 

Cawwnts 


Result 
tteMrit 


Audltort CoiMnts 


Fernberf 


30162 


A 


Z 




S«npler imI function, tvo day sample, 

March 7-9, 1982 

Sample yolume • 5OO40 litres. All 

appropriate codes and coiments 

appended. 


95480 


1 

1 






Suspected filter contamination, Teflon 
filter dropped while unloading. This 
was a passive filter - analytical 
results from LIS I1-HH4* • !EF (egulp- 
■ent failure) M-NO-, • .25 <¥, SO. ■ 
1.25 <W. 

All appropriate codes and coaments 
appended. 


90152 


E 






Suspected filter pack leak. Sample 
frOB March 13 08:00 hrs, to March 14 
09:00 hrs 1982. 


LongMOods 


15342 

15343 
15345 
15362 
15364 






U 


Suspected contamination of nylon 
filters all N-HNO3 results should b* 
flagged to reflect this. 


15347 


H 


z 




Two day sample, February 14-16, 1982 
known contamination of nylon filter - 
not flagged as unreliable. N-HNO3 
result should be flagged. 


15352 


G 






Technologists coament on field sheet - 
"twist lock nozile cracked at inlet 
port - Incorrect volume reported 23920 
litres'. Appropriate comments 
appended. 


15501 









Technologist coonent on field sheet - 
Tygon tubing separated at coupler 
Joint - volume reported Is actual 
sample volume'. Appropriate comments 
appended. 
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TABLE 3.10 

Gross Limit Checks - Lower Limit 

Feb., Mar., June, July 1982 
Fernberg and Longwoods 









Concen- 














Sample 


tratlon 


Result 


Field 


Office 




Site 


Parameter 


Number 


ug m-5 


Remark 


Conment 


Cofnment 


Auditor's Conment 


Fernberg 


so, 


30165* 


0.00 








P>A 


- ■>• a 


II 


30152 


" 








P>A 


" 


H 


10677* 


•1 








P>A 


■■ 


« 


95433* 


M 








P>A 


II 


" 


95441* 


" 








P>A 


M 


M 


95449* 


H 








P>A 


41 


.. 


95455 
95457* 


II 








P>A 
P>A 


« 


w 


95459 










P>A 


H 


II 


95461 


U 








P>A 


" 


II 


95465* 


H 








P>A 


n 


« 


95473* 


" 








2nd Passive Value UI! 


II 


II 


95481* 


" 








A>P 


H 


" 


95482 










LIS not available 
to auditor 


n 


M 


95489* 


n 








P>A 


H 


M 


15471* 


« 








P>A 


Longwoods 


M 


15475 


" 








P>A 1 


a 


H 


15487 


II 








P>A 


» 


» 


15501* 


" 




D 




l$t Passive Value UI! 


" 


" 


15512 


» 








P>A 


" 


" 


15517* 


** 








P>A 



* - Port # 1/8 
UI! "no data, undetermined interference (laboratory codel 
P>A ■ passive result greater than active 
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TABLE 3.10 (cont'd) 

Gross Limit CHecks - Lower Umlt 

Feb.. Mar., June, July 1982 
Femberg and Longwoods 









Concen- 














Sample 


tration 


Result 


Field 


Office 




Site 


Parameter 


Number 


ug in-3 


Remark 


Comment 


Comment 


Auditor's Comment 


Fernberg 


N-NO3 


30152 


0.00 


<T 










n 


30133* 
30135 


II 


<T 






<W on LIS output P>A 


•1 


•* 


30136 


w 


<T 








.. 


" 


95425* 


It 








P>A 


II 


H 


95433* 


u 


<T 








■■ 


H 


30124* 




<W 








Longwoods 


•* 


15352 


0.00 




G 




<T on LIS output 


w 


" 


15353 


II 








<W on LIS output 


u 


H 


15354 


•• 








P>A 


n 


H 


15355 


» 








P>A 


u 


M 


15358* 


11 








P>A 


Fernberg 


N-HNO3 


95425* 


III 








P>A 


m 


m 


95426 


" 








<T on LIS output 


Longwoods 


so,= 


15471 


It 








Should be 0.23 ug m*' 


» 


H 


15358* 


H 








P>A 


ti 




1 54 96 


" 








P>A 



* - Port # 1/8 

<T » reported at the detection limit 
<W « no response, minimum possible result recorded 
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TABLE 3.11 










Gross Limit Checks - Upper Limit 




Feb., Mar., June, July 1982 
Fernberg and Longwoods 










Concen- 














Sample 


tration 


Result 


Field 


Office 




■ Site 


Parameter 


Number 


ug m-3 


Remark 


Comment 


Comment 


Auditor's Comment 


iernberg 


N-NO3 


95428 


0.09 








( 

1 


H 


II 


95460 


0.26 










H 


SO2 


30174 


16.96 










longwoods 
II 


11 

SO, 


15497 
15347 


31.85 
14.93 


U 


H 


Z 


Nylon filter-H 


H 


II 


15503 


40.93 










* 


N-NH, 


15344 


9.057 


> 








1 B 


11 


15345 


9.426 


> 


I 




Nylon Filter- I 


H 


II 


15346 


9.509 


> 








II 


II 


15347 


4.744 


> 


H 


Z 


Nylon Filter-H 


H 


II 


15348* 


8.944 


> 








M 


II 


15362 


9.140 


> 


I 




Nylon Filter- I 


1= Port # 


1/8 
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• If an active result wUh a remark code 1s less than a passive 
result, the active 1s reported as "0" and the remark code Is 
removed. However, for samples 15352, 15353 and 95426 the 
result remark codes (<T, <W) appended to analytical results on 
the L.I.S. output did not appear 1n the Dally Ambient A1r 
Concentration Listings {MOE, 1984), 

• With the exception of one Instance, all exceedances of the 
upper limits were not flagged as unreliable. It should be 
noted that all exceedences of the upper limits were studied by 
a staff scientist. 

• For all Instances where the passive result was equal to or 
exceeded the active result a zero was reported. 

5. Check - Passive Correction Calculations. This Involved deter- 
mining if the passive correction calculations were executed properly. 
Filters in sampling ports 2 to 7 are corrected using the equation shown 
below. Twenty-five percent of data from the study period corrected 
using this calculation was spot checked. 

, „ Active Result -6/7 Passive Result (wg fH" ) 
Reported^Conc. ^•^- 3 

(ug m' ) Sample volume (m ) 

Switched filters (ports 1 and 8) are left in the field for a 
two-week period and are corrected using the equation shown below. All 

( nin I ltd N inililK i ii()' 'iiH:' 11" 
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switched filters from the study period were examined for passive correc- 
tion errors. 



Active Result - P^ - 6/7 P^ (yg flT^) 

Reported^Conc. = —j- 

(yg m" ) Sample Volume (m ) 



where: 



P, = Passive Result from week 1 
Pp - Passive Result frow week 2 



If a passive result is: 

1. reported at the 'detection limit (<T) then a value correspond- 
ing to one half the detection limit is utilized for passive 
correction. 

2. reported as no response, minimum possible result reported (<W) 
then a value of zero is utilized for passive correction. 

3. equal to or exceeding the active result then the concentration 
is reported as equal to zero. 

Findings - A few discrepancies {four samples) between the 
reported concentration and calculated concentration for switched filters 
were observed and tabulated (Table 3.12). The samples appear to have 

Concord ScientiiK toicxM.Uion 



TABLE 3.12 
Calculation Errors - Switched Filters 



r, 











1st 


2nd 








Sample 
No. 


Volume 
m3 


Parameter 


Active 
Result 


Passive 
Result 


Passive 
Result 


Calculated 
Cone. 


Reported 
Cone . 


Comments 








vg fil-i 


ug fil-^ 


vg fll-^ 


ug m"3 


ug m~3 




30155 


23.58 


N-NO3 


1.05 


.25 < W 


.25 < T 


0.04 


0,06 


associated 
passive 
numbers not 
recorded 
on field 
data sheets 


15470 


26.36 


N-NH^ 


5.10 


0.70 


.25 


0.159 


0.171 


only 1 
associated 






N-HNO3 


8.25 


5.25 


2.50 


0.03 


0.14 


passive 
number 






SO, 


60.05 


60.05 


33.35 


0.00 


0.33 


recorded 
on field 
data sheet 


15478 


26.37 


so,= 


170.00 


1.25 <W 


1.25<W 


6.41 


6.45 


H 






N-HNO3 


58.75 


2.50 


5.00 


1.97 


2.15 








so. 


247.00 


33.55 


56.70 


6.25 


8.29 




15993 


18.63 


so^= 


335.00 


1.25 


3.75 


18.82 


18.88 


W 






N-HN03 

1 


25.20 


2.50 


4.20 


1.04 


1.16 





CO 
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been corrected using the wrong passive filter results. Field data 
sheets for passive and active samples were examined and the associated 
passive sample numbers for these switched filters were determined. 
These "switched" errors were found to have originated from the computer 
program which assigns passive corrections to active results. Because 
these four samples had "unusual" passive dates, the errors went un- 
noticed. The error would have been eliminated if the passive numbers 
themselves, recorded on the field sheets, were used rather than passive 
dates (presently being used in the networks for validation purposes). 

While performing passive calculation checks, it was noted that 
In some instances an average passive value was determined and the active 
concentration calculated and reported (Table 3.13) when: 

a) a passive filter result was missing due to laboratory equip- 
ment failure or undetermined interference; or 

b) one of the passive filter results exceeded the active filter 
result; or 

c) a passive sample was not received at the laboratory. 

This procedure, which was carried out at the discretion of the database 
manager, should be documented and included in the Data Management 
section of the APIOS Technical and Operating Manual. When reporting 
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TABLE 3.13 
Blank Corrections Using an Average Passive Value 



Sample 
No. 


Volume 
ra3 


Parameter 


Active 

Result 

ug f11-» 


1st 
Passive 
Result 
wg fll-^ 


2nd 
Passive 
Result 
pg fll-^ 


Reported 

Cone. 

gg m~3 


Comments 


95473 


22.34 


M-NH^+ 


2.30 


0.05<T 


!EF 


0.100 


!EF = laboratory equipment 
failure 


15374 


27.72 


N-NH,* 


50.00> 


!EF 


!EF 


1.774> 


!EF = laboratory equipment 
failure 


15390 


28.75 


N-NH^* 


48.50 


!EF 


0.75 


1.649 


!EF = laboratory equipment 
failure 


15390 


28.75 


so,= 


570.00 


30.00 


NR 


18.19 


NR = laboratory did not 
receive sample 


15342 


28.22 


SO2 


420.00 


20.00 


580.00 


13.17 




15348 


27.74 


SO2 


23.30 


580.00 


13.30 


19.96 




15493 


18.63 


SO2 


220.00 


76.70 


UI 


9.51 


UI = undetermined 
Interference 



J. 
j 
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these data the office comment code "P" (passive filter missing - average 
passive result used as blank correction) or an equivalent result remark 
should have been appended. 

6. Check - Passive results for the Longwoods and Fernberg sites 
were tabulated by season for the study period. The mean, standard 
deviation, range and variance were calculated for each and the data 
examined. 

Findings - (see Tables 3.14 to 3.17) Results with an asterisk 
(*) indicate that the passive result was greater than an associated 
active result. The passive SO2 results are extremely variable and all 
SO passive results from Fernberg during June and July (1982) were 
greater than the active results. 

Samples No. 15389 and 15397 have SO^ results reported as 0.25 
ug filter''^ which 1s less than the detection limit of 1.25 ug filter . 

The variability of passives results should be determined and 
accounted for in the calculation. 

3.4 Performance Audit of Data Handling and Validation Procedures - 
Cumulative (28-day) Air Concentration Data 

Specific data validation procedures have been outlined (Chung, 
1984) to screen cumulative ambient air monitoring results. These proce- 
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TABLE 3.14 
1982 Passive Results for Longwood (June, July) 



Passive 
ID 


Date 
Off 


Associated 
Active 

1 
II 


NH. 
Mg fll-^ 


NO, 

ug fiT^ 


SO, 
ug fil-i 


HNO3 

ug fir^ 


SO, 

ug fii-^ 
(Nylon) 


SO, 
pg f1l-» 


Coimnent 


Detection Limits 




0.05 


0.25 


1.25 


0.25 


2.5 


3.35 




15469 


2/6/82 


15462 to 
15470 


.70 


.25<W 


1.25<W 


5.25 


36.25 


60.05* 


*P > A 


15477 


9/6/82 


15470 to 
15478 


.25 


.25<W 


1.25<W 


2.50 


22.50 


33.55 




15485 


15/6/82 


15478 to 
15486 


.25 


.25<W 


1.25 


5.00 


22.50 


56.76 




15492 


22/6/82 


15486 to 
15493 


.10 


.25<W 


1.25 


2.25 


21.20 


76.70 




15500 


13/7/82 


15493 to 
15501 


.10<T 


.25<W 


3.75 


4.20 


31.20 


!UI 


undetermined 
Interference !UI 


15508 


20/7/82 


15501 to 
15509 


.40 


.25<W 


1.25 


3.00 


27.50 


53.40 




15516 


27/7/82 


15509 to 
15517 


.05<T 


.25<W 


1.25<W 


2.50 


16.20 


103.00* 


* P > A 






Mean 


0.26 


.25 


1.61 


3.53 


25.33 


63.91 








Std 

Deviation 


0.23 





0.95 


1.26 


6.77 


23.62 








Variance 


0.05 





0.89 


1.60 


45.80 


55 








Range 


0.65 





2.50 


3.00 


20.05 


69.45 





I 

m 



1982 Passive Results for Longwood (February. March) 






Passive 
ID 


Date 
Off 


Associated 
Active 


yg fil-^ 


NO3 
ug fil-^ 


so^ 
ug fll-i 


HNO3 

ug fii'^ 


so^ 
ug fil-^ 
(Nylon) 


SO, 
ug fll-^ 


Comraent 


Detection Limits 




0.05 


0.25 


1.25 


0.25 


2.5 


3.35 




15341 


9/2/82 


15334 to 
15342 


.25<T 


.25<T 


2.50 


5.25 


28.80 


20.00 




15349 


16/2/82 


15342 to 
15350 


.25<T 


.50 


2.50 


8.25 


32.50 


580.00* 


*P > A 


15357 


23/2/82 


15350 to 
15358 


0.45 


1.75* 


8.75* 


2.75 


37.50 


13.30 


*P > A 


15365 


2/3/82 


15358 to 
15366 


!EF 


.25 


1.25 


1.25 


20,00 


10.00 




15373 


9/3/82 


15366 to 
15374 


!EF 


1.00 


3.75 


3.50 


28.80 


26.70 




15381 


16/3/82 


15374 to 
15382 


!EF 


.50 


1.25<T 


3.00 


36.20 


33.30 




15389 


23/3/82 


15382 to 
15390 


!EF 


.25<T 


* .25<T 


2.50 


28.80 


30.00 


*less than de- 
tection limit 


15397 


1/4/82 


15462 to 
15470 


.75 


.25<W 


* .25<W 


3.00 


27.50 


!NR 


*less than de- 
tection limit 






Mean 


0.42 


0.59 


2.56 


3.69 


30.01 


101.9 








Std 

Deviation 


0.23 


0.53 


2.77 


2.15 


5.50 


211 








Variance 


0.05 


0.28 


7.69 


4.63 


30.21 


445 








Range 


0.50 


1.50 


8.50 


7.00 


17.50 


570.00 





I 
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TABLE 3.16 
1982 Passive Results for Fernberg (June. July) 



Passive 
ID 


Date 
Off 


Associated 
Active 


pg nl-^ 


NO, 
ug fil-i 


SO. 

Vig fil-^ 


HNO, 
ug fil-^ 


SO, 
gg fll-i 
(Nylon) 


SO, 
pg fil-» 


Conine nt 


Detection Limits 




0.05 


0.25 


1.25 


0.25 


2.5 


3.35 




95432 


8/6/82 


95425 to 
95433 


.05<T 


.25<T 


1.25<T 


1.00 


3.75 


50.00* 


* P > A 


95440 


15/6/82 


95433 to 
95441 


.05<T 


.25<T 


1.25<T 


.25<T 


1.25<T 


63,40* 


* P > A 


95448 


22/6/82 


95441 to 
95449 


.05<T 


.25<W 


1.25<W 


.25<W 


1.25<W 


5.67* 


* P > A 


95456 


29/6/82 


95450 to 
95457 


.05<T 


.25<W 


2.50 


.75 


3.75 


66.7* 


* P > A 


95464 


6/7/82 


95457 to 
95465 


.05<T 


.25<W 


1.25<W 


.50 


2.50 


53.40* 


* P > A 


95472 


13/7/82 


95465 to 
95473 


.05<T 


.25<W 


1.25<W 


.50 


1.25<T 


40.00* 


* P > A 


95480 


20/7/82 


"95473"to'" 
95481 


!EF 1 


.25<Wl 


1.25<Wi 


.75 


3.75 


!UI 


1 Teflon 
filter dropped 


95488 


27/7/82 


95481 to 
95489 


.10<T 


.25<W 


1.25<W 


.25<W 


2.50 


30.00* 


* P > A 


95496 


5/8/82 


95489 to 
95497 


.10<T 


.25<W 




1.25<W 


1.00 


2.50 


23.30* 


* P > A 






Mean 


0.0625 


.25 


1.38 


0.58 


2.50 


47.94 








Std 
Deviation 


0.023 





0.42 


0.31 


1.08 


15.55 








Variance 


0.005 





0.174 


0.09 


1.17 


242 








Range 


0.05 





1.25 


0.75 


2.50 


43.40 





t 

is* 



§ 



I 
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TABLE 3.17 
1982 Passive Results for Femberq (February. March) 

















r- 






Passive 
ID 


Date 
Off 


Associated 
Active u 


MH, MO 3 
g fiT' ug f11"' t 


g fll-i I 


HNOj SO^ 
9 fll-i ug fiT' w 
(Nylon) 


SO, 
g fll-i 


Comment 


Detectio 


T Limits 




0.05 


0.25 


1.25 


0.25 


2.5 


3.35 




30132 


2/2/82 


30130 


.20 


.25<T 


1.25<1 


1.50 


11.20 


10.00 




30117 


9/2/82 


30134 to 
30118 


.20 


.25<T 


1.25<W 


.75 


6.25 


13.30 




30180 


23/2/82 


30173 to 
30165 


.05<M 


.25<W 


1.25<W 


.25<W 


20.00 


26.70* 


* P > A 


30172 


2/3/82 


30165 to 
30171 


.15 


.25<W 


1.25<W 


.75 


10.00 


16.70 




30164 


9/3/82 


30157 to 
30163 


.25 


.25<W 


1.25<W 


1.00 


8.75 


13.30 




30156 




16/3/82 


30149 to 
30157 


.30 


.25<T 


1.25<T 


.75 


15.00 


70.00 




30140 


23/3/82 


30133 to 
30138 


.30 


.25<T 


1.25<W 


.25 


10.00 


53.40 




10684 


30/3/82 


10678 to 
10683 


.05<T 


.25<U 


1.25<H 


.75 


30.00 


277.00* 


* P > A 






Mean 


0.19 


.25 


1.25 


0.75 


13.90 


60.05 








Std 
Oeviation 


0.10 








0.40 


7.75 


90.30 








Variance 


0,01 








0.161 


60.0 


8154 








Range 


0.25 








1.25 


23.75 


267.00 





I 
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dures consist of 1) an integrity check of sample analysis and 2) statis- 
tical screening of the data for outliers using the Dixon Ratio Test. 

In the performance audit non-routine manual checks were 
performed to determine if the data validation procedures were being 
utilized properly. The audited data set consisted of data collected at 
five sites (one site in each of the five regions): Gowganda, Shallow 
Lake, Uxbridge, Pickle Lake and Smiths Falls for the period January to 
December of 1982. A description of the checks performed along with the 
findings are described below. 

1. Check - Field sheets were compared to operator sheets to 
ensure that all information recorded on the operator sheets was trans- 
ferred and that it was transcribed correctly. The field sheets were 
also checked for proper changeover dates and proper use of field codes. 

The field sheets contain all information pertinent to the 
samples submitted such as the sampling period, filter type, any problems 
observed, volume readings, etc. Field and operator sheets were avail- 
able for all sites with the exception of Pickle Lake (operator sheets 
were unavailable). 



Concoril xii-ntiik 1. nrin 'I.iMiHI 



- 3.53 



Findings - A summary of the discrepancies found between the 
field and operator sheets is shown on Table 3.18. In most cases the 
discrepancies are due to errors in information transfer or missing 
information. It is recommended that the technologists be made aware of 
these types of discrepancies and that a system of automatic verification 
be introduced which would prevent these inaccuracies. This system was 
also recommended in the Phase I audit (CSC, 1984) and involves comparing 
a portion (10^) of the operator and field sheets every six months. This 
would be the responsibility of the QAC. When a discrepancy is found, 
the technologist is made aware of it and a revised field sheet 
submitted. 

2. Check - Reported data against field sheets. The reported data 
was checked manually against the field sheets to ensure all data were 
transcribed correctly. Field sheets and reported data were available 
for all of the five sites and data for 60 samples were examined. 

Findings - A summary of the discrepancies found between the 
field sheets and reported information is found in Table 3.19. For the 
most part, information transferred from field sheets to the database was 
correctly executed with the exception of the data from the Uxbridge site 
{12 transcription errors were noted). A system of automatic 
verification should be implemented to minimize this type of problem. If 
complete and accurate data are to be reported, transcription errors and 
omissions must be eliminated. The QAC should continue to spot check 
field sheets and database output particularly from the Uxbridge site. 
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TABLE 3.18 
Discrepancies Between Field and Operator Sheets 







Number of Occurrences 






Information 














Not Trans- 






Improper 








ferred From 






Code Used 






# of Data 


Operator 


Information 


Sample 


for Filter 


Filter Lot 




Sheets 


Sheets to 


Transferred 


Changed on 


Type or SIS 


Number Not 


Site 


Examined 


Field Sheets 


Incorrectly 


Wrong Day 


Sample Type 


Recorded 


Gowganda 


12 


8 


5 


4 


1 ■ 


10 


Uxbridge 


14 


2 


1 


7 


3 


11 


Shallow 


12 


4 


1 


1 



1 





Lake 


1 








1 




Smiths Falls 


14 


4 





3 


1 


10 



Note: Pickle Lake excluded; operator sheets were not available. 
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TABLE 3.19 
Discrepancies Between Field Sheets and Reported Data 







Number of Occurrences 






Volume not 


Field Codes 


Information 


Incorrect 






Recorded 


not Trans- 


Transferred 


Time Trans- 








ferred from 


Incorrectly 


cribed on 




# of Data 




Field Sheet 


From Field 


Database 




Points 




to Database 


Sheets to 


Output 


Site 


Examined 






Database 




Gowganda 


12 











1 


Uxbridge 


12 





1 


12 




Shallow 


12 





1 








Lake 












Smiths Falls 


12 


1 








1 


Pickle Lake 


12 





1 


3 


3 
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3. Check - Summary of field and office comments. Field, office 
and laboratory comments for cumulative ambient air monitoring network 
for the period of January to December 1982 were examined. 

Findings - Table 3.20 summarizes the frequency of field, 
office and lab comments. To determine the frequency of occurrences per 
site due to hydro failure, sample contamination or flow volume problems, 
field comment codes associated with each of these problems were combined 
and a relative frequency was calculated. The results are tabulated In 
Table 3.21. 

4. Check - Data Validation Procedures. Procedures outlined in 
Data Validation Procedures, Cumulative Ambient Air Monitoring Network 
(Chung 1984) were examined to determine whether they are being implemen- 
ted properly. Validation procedures utilized consist of two approaches. 
The first approach consists of an integrity check in which the sample 
flow, volume and sampling period are examined and the second is a 
statistical screening of the data for outliers (using the Dixon Ratio 
tests). 

All reported data from the five sites selected during the 
period January to December 1982 were screened to ensure proper tags 
(i.e. Z for sampling period, F for flow volumes) were applied to 
samples that were not within specified guidelines and 25% of the 
reported data were randomly checked using the Dixon Ratio test. 
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TABLE 3.20 
Frequency of Field and Office Comments - Low Vol Data (1982) 



































Lab Comments 




Office 


Result Remark 


Site 


Number of 
Records 


Field Coiiwents 


Comments 


Code 


A 


B 


C 





E 


F 


a 


H 


1 


J 


K 


F 


Z 


X 


• 


> 


<T 


<W 


A 


U 


Gowganda 


12 


























4 




5 


1 








I 


Uxbridge 


12 






1 




















2 




11 




1 


2 




3 


Shallow 


12 


3 






1 




2 




1 


3 






1 


1 




22 




1 








Lake 












































Smiths Falls 


12 


1 


3 












2 


2 






2 


3 




30 


1 








2 


Pickle Lake 


12 


1 1 










5 










3 


6 




23 


2 


4 1 




' 



G - Flow Line Problems 
H - Known Contamination 
I - Suspected Contamination 
J - Heavy Dew or Fog 
K - Sample Not Submitted 



Field Cowment Code Inden 

A - Sampler Malfunction 

B - Known Hydro Failure 

C - Suspected Hydro Failure 

D - Known Filter Pack Leak 

£ - Suspected Filter Pack Leak 

F - Gasmeter Not Equal to Rotameter 

Office Comment Code Index 

F - Data Invalidated - flow volume not within upper and lower limits (150% and 50% of 

2880 litres/day respectively) 
Z - Sampling period abnormal - sampling period not within upper and lower limits izy 

and 27 days respectively) 

X - Sample lost 

Result Remark Code 

> - Actual result greater than value reported 

< - Actual result less than value reported 

<T - Actual result less than criterion of detection 

<W - No response, minimum possible result reported 

A - Approximate value 

U - Unreliable result 

* - Mo data reported 
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TABLE 3.21 



Frequency of Field Consents Per Site for Hydro Failu re. Flow Volume 

Problems and Contamination 
Cumulative Air Sampling Network (1982) 



Site 


% 
Hydro Failure 

B+C 


Fl 


% 
ow Volume 

D+E+F+G 




% 

Contamination 
H+I 


Gowganda 








Uxbridge 


8 







Q 


Shallow Lake 


33 




25 


33 


Smiths Falls 


25 




17 


33 


Pickle Lake 


17 




33 
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All samples had the proper office comment codes appended. 
Samples which exceeded the upper and lower limits for flow volume (150% 
and 50% of 2880 litre/day, respectively) were appended with the code 
"F". Samples which exceeded the upper and lower limits of the standard 
sampling period (29 and 27 days, respectively) were appended with the 
code "Z". 

The Dixon Ratio test was applied to approximately 25% of the 
data from January to December 1982. It was found that it is being used 
correctly to identify outliers in the reported data. 
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4. SYSTEMS RELATED AUDIT 

A systems related audit involves reviewing network operations 
and systems to determine how well they satisfy the objectives of a moni- 
toring network. The primary objectives of the APIOS monitoring network 
are the quantification of both wet and dry deposition patterns across 
the province of Ontario (Chan et al., 1984, 1st revised edition). 

Since the Phase I systems audit reviewed, in detail, the over- 
all operating system and QA system of the APIOS network, the Phase II 
system audit concentrated on an evaluation of the daily air concentra- 
tion data reporting and validation procedures, recent changes to the 
network organization, and routine QA/QC procedures with respect to QA 
objectives. 

4.1 Daily Air Concentration Data Validation Procedures and 

Reporting 

The APIOS data validation procedures for the daily ambient air 
monitoring network have recently been documented (Kirk, 1984) and should 
be included in the Data Management section of the APIOS Technical and 
Operating Manual {Bardswick, 1983). 

Basically, these procedures involve two editing/screening 
stages. The first stage involves comparing the database output, in 
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which omissions or inconsistencies have been noted (e.g. missing sample 
volumes, abnormal sampling periods, dates, times, etc.), to the field 
data sheets. This step ensures completeness and accuracy of the data 
set. The second stage utilizes statistical computerized screening tech- 
niques such as gross limit checks and principal component analysis to 
select unlikely analytical results. These selected results are then 
reviewed by the database scientist who subjectively decides whether 
results appear reasonable or not. The data validation system seems to 
be well conceived and implemented, however, modifications are suggested 
in qualifying and reporting data. 

Use of Comment Codes 

As discussed in the performance audit, data with the field 
comment codes I (suspected contamination) and H (known contamination) 
often provide misleading information about the quality of the data set. 
This has been noted when the codes apply to one filter in particular as 
opposed to the entire set of filters comprising one sample. When this 
occurs, an appropriate result remark code should be appended to the 
analytical results of the filter rather than appending the field comment 
code to the entire data set for the sample. 

Inconsistencies in the use of these codes has also been noted 
where one technologist has used the code I, while another uses the code 
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H for the same situation, e.g. "dropped nylon filter on floor while 
unloading filter pack". 

The use of these codes (H and I) should be standardized. Any 
observations which could affect the integrity of any filters in the 
filter pack should always be included on the field data sheets. These 
observations have proven to be quite valuable to the database scientist 
when qualifying data with field comment codes appended. 

The letter "P" is utilized as an office comment code and a 
result remark code. However, the definitions associated with these 
codes differ. The office comment code "P" Indicates that an average 
passive result was used for blank correction and the result remark code 
"P" indicates that a sample was not corrected with a passive result. 
Because these codes have different meanings the same letter should not 
be used. It should be noted, however, that for the 1982 data the office 
coimient code "P" was never used in the data validation procedure. 
Therefore, it should have been removed from the 1982 report which 
describes these codes. 

Symbols utilized as office comments codes, field comment 
codes, and result remark codes should be unique and have precise 
definitions to avoid ambiguity in Interpretation. 
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When a passive result 1s not available because of laboratory 
equipment failure or undetermined interference, an average passive 
result 1s utilized for passive correction. When this occurs for a 
particular parameter in the data set, a result remark code equivalent to 
the office comment code "P - passive missing, average passive result 
used for blank correction" should be appended. 

The procedures or guidelines utilized by the database 
scientist in determining whether an active result will be corrected 
using an average of passive values should be documented and included in 
the Data Management section of the APIOS Technical and Operating Manual 
(Bardswick, 1983), as well as Data Validation Procedures (Kirk, 1984). 

Reporting Data 

Currently, the APIOS network reports data in the form of two 
or three significant figures for air concentration measurements. The 
data are calculated by subtraction of a weekly passive filter value from 
the corresponding actively sampled integrative value for each parameter. 
Active values less than or equal to the passive value are reported as 
zero. 

The performance audit section of this report reveals that in 
many Instances the passive result was greater than the active result. 
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It is recommended that re-evaluation of passive correction is in order 
to determine why passive results are frequently greater than the active 
results. 

The reporting of data should ensure that results are not 
arbitrarily truncated nor set to zero if active filter results give 
negative numbers after correction for passive filter contribution. The 
level of confidence with low level data should be reported to data users 
in the form of codes attached to specific numbers outlining measurement, 
method detection criteria and detection limits, and the statistical 
levels of confidence in numbers. (See the IJC Bulletin on detection 
limits. Appendix III.) 

4.2 Network Organization and Operation 

The recent change to regional ization of field operations has 
an important impact on the quality of the data collected and produced by 
the network. From an operational point of view, it will ease logistical 
constraints of field operations by allowing the APIOS technical people 
access to other ministry resources. It should also improve many aspects 
of the field program by providing additional support to APIOS field 
activities, particularly in the remote areas of the province because 
other ministry employees will be able to assist in sample collection and 
maintenance. 
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However, APIOS and regional managers nnjst take appropriate 
action to ensure that APIOS technical staff concentrate their efforts on 
the APIOS programme to ensure continuity and comparability within and 
between regions. 

Reduction in the technologists travelling time (by co-ordina- 
ting site visits with other ministry activities) should improve staff 
morale and allow technologists to spend time on other aspects of the 
APIOS network such as processing and interpretting QA/QC information and 
data validation procedures. Regionalization will require work on the 
part of the APIOS head office staff in terms of support, organizational 
requirements, co-ordination and quality control. 

The emphasis on network QA/QC must continue. QA/QC and over- 
all network operating systems must Improve in order to implement the 
proper QA/QC monitoring procedures to ensure the system is in control. 
It will be the responsibility of the QA Coordinator to ensure that the 
field personnel are meeting their APIOS field requirements by designing 
and implementing a system to monitor field QA/QC procedures. In 
particular, his functions with respect to corrective action monitoring, 
routine QA/QC assessment and QA/QC reporting and interpretation must 
expand significantly. 

A sound system for co-ordinating APIOS activities in all 
regions should be employed. Regional managers {A1r Quality Chiefs) 
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should be involved in the relevant aspects of field programme planning, 
and a mechanism should be implemented for incorporating their input into 
the network operating system. It is recommended that regional air 
quality chiefs and APIOS Network managers meet on an annual basis to 
discuss these aspects of the programme. 

Recent staff turnovers in the APIOS programme have had 
significant negative impacts on the operations of the field programmes 
in the northeastern and southeastern regions. Attempts to replace 
full -time technologists with part-time consulting personnel were "band 
aid" solutions to staffing problems that would have been effective in 
the short-term (i.e. less than one month), but were totally inadequate 
during long-term periods of eight and seven months at Kingston and 
Sudbury, respectively. Consulting personnel were asked to collect and 
process samples and "maintain" site operations in the time allocated 
(four days per week in the Kingston region and two weeks per month in 
the Sudbury region). Some problems arising from this situation 
included: 

poor communication between operators and network personnel 
which led to inadequate instrument servicing; 

missed changeovers; 

late spring instrument changeover which resulted in a loss of 
storage gauge data; 
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poor communications between operator and consultants because 
of time restraints in the field. 

Required instrument servicing was performed as quickly as 
possible to minimize downtime and loss of data, however, complete 
preventive maintenance was not carried out due to time constraints 
placed on the consultants. In order to avoid this type of problem in 
the future, it is recommended that a time limit of one month be put on 
replacement of network personnel and that network management support 
this time limit vigourously to help maintain the quality of the network 
data. 

4.3 Evaluation of Routine QA/QC Procedures with Respect to QA 
Objectives 

Routine QA/QC monitoring has now been implemented in many 
areas of the program and has been evaluated for the cumulative network 
for the period June 1980 to December 1981 (Bardswick, 1984). In this 
report information on data recovery, reasons for loss of data, labora- 
tory QA/QC and special studies were reported. 

All QA/QC procedures which have been implemented should be 
reported, and it is essential that these reports be prepared on an 
annual basis as quickly as possible, in order that the information 
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contained in them may be used in the annual network review sessions 
described in the QA Manual {CSC, 1984b). 

Since data from the recently implemented routine QA procedures 
1s not yet available, it was not possible in this audit to quantify data 
quality. 
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5. 



• RECOMMENDATIONS SUMMARY AND PHASE I AUDIT FOLLOW-UP 



This chapter summarizes, in point form, the recommendations 
made in this report (in the order In which they appear), as well as 
reviewing the APIOS actions on the recownendations from the Phase I 
audit. The section 1n which the recommendations originate is referenced 
in brackets. Items which may significantly affect data quality have 
been identified by an asterisk (*) and the QAC should review these 
Immediately and take the appropriate action. Other recommendations 
should be acted on as soon as possible. 

* 1. Corrective action at audited sites (Secton 3.1; Appendix II). 

* 2. Review sites with a marginal or poor rating (Section 3.1). 

3. Post "Procedural short lists" at monitoring sites (Section 
3.1). 

4. Inconsistencies in instrument configuration from region to 
region should be removed (Section 3.1). 

5. Service MIC samplers at Dorset (Section 3.2). 

* 6. Update network site documentation for the sites in the Central 
Region (Section 3.2). 
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7. Regional technologists should continue to maintain close 
communication with operators, encourage operators to contact 
them for any reason and try to co-ordinate their site visits 
so that they can meet with the operators as often as possible 
(Section 3.2). 

* 8. Technologists should carry out procedures audit at least twice 

a year (Section 3.2). 

9. Operators should fill out operator sheets, monthly site 
summaries and log-books in duplicate, one copy should be 
retained for future reference and a second copy forwarded to 
regional technologist (Section 3.2). 

* 10. Quality Assurance Activity Log Sheet should be developed and 

implemented in network operations and procedures (Section 
3.2). 

* 11. Retraining sessions for technologists should be carried out at 

the next Quarterly APIOS Technologist meeting (Section 3.2). 

* 12. Technologists should ensure that they have on hand at least 

the reconriended instrument spare parts at all times (Section 
3.2). 
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* 13. Instruments in the Central region should be calibrated and 

serviced on a routine basis (Section 3.2). 

* 14. Technologists should visually check field data sheets for 

omissions and errors (Section 3.3.1). 

* 15. Procedures or guidelines followed by the database scientist, 

used for correcting active values and an "average passive 
value", should be reviewed and documented and an appropriate 
code appended to the reported concentration (Section 3.3.1). 

16. Procedures or guidelines used in (15) should be included in 
the Data Management section of the APIOS Technical and 
Operating Manual (Section 3.3.1). 

17. The method of correction for passive filter results should be 
reviewed (Section 3.3.1). 

18. Air concentration data validation procedures should be inclu- 
ded in the Data Management section of the APIOS Technical and 
Operating Manual (Section 3.3.1). 
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19. QAC should spot check 10% of the operator and field sheets 
every six months to ensure accuracy and completeness of 
reported data (Section 3.41. 

20. QAC should continue to spot check field sheets and database 
output (Section 3.4). 

21. Result remark code indicating potential or known contamination 
of a particular filter should be incorporated into the coding 
system (Section 4.1). 

22. The use of the field comment codes H and I should be standard- 
ized (Section 4.1). 

23. Observations which could affect the integrity of any filters 
should always be Included on the field data sheets (Section 
4.1). 

24. Symbols used for office comments, fields comments and result 
remarks should be unique and have precise definitions to avoid 
ambiguity in interpretation (Section 4.1). 

25. The level of confidence in the low level data should be 
reported to data users in the form of codes attached to 
specific numbers, outlining measurement method detection 
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criteria and detection limits and the statistical levels of 
confidence in numbers (Section 4,1). 



26. Regional air quality chiefs and APIOS network managers should 
meet on an annual basis to discuss these aspects of the 
I program (Section 4.2). 



* 27. QA/QC reports should be generated for all data sets and QA/QC 
information should be included in these data sets (Section 
4.3). 

Phase 1 Audit Recommendations and the Actions Taken 

The following describes the actions taken in response to the 
47 recommendations generated from the Phase I audit findings during 
1983-84. 



I #1 Corrective action at audited sites. 

_ action Monitoring sites that required corrective action have been 

■ Investigated by the QA Coordinator and the environmental tech- 

I 
I 
I 

^ Com ord SiifnliiJi C'ofi'nir.iiiiin 

I 



5.6 



nologlst responsible for the site. Where possible, changes 
have been made to Improve the site (i.e. remove a tree, 
upgrade operator, etc.). A technologist meeting was held in 
which all corrective actions were discussed and what steps 
would be taken by the technologists to address the corrective 
actions recommended by the auditor. 

#2 Review of marginal or poor siting by QA Coordinator. 

action A review of all sites was undertaken by the QA Coordinator. 
Using site documentation, photographs, and all of the 1982 
data collected at each site, an evaluation of the representa- 
tiveness or non-representativeness was determined for every 
site. From this evaluation, the Milton and Attawapiskat moni- 
toring sites were removed from the networks. Colchester and 
Moonbeam, both rated "poor" by the auditors are still consid- 
ered representative monitoring sites by APIOS personnel. 

#3 Feedback to operators on networks. 

action Network operators are kept abreast of monitoring activities by 
way of a semi-annual newsletter. Also, annual reports of 
chemical listings for every site are provided to every 
operator. 
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#4 Update and upgrade training for operators. 

action All technologists have been instructed to thoroughly train 
every operator 1n operating procedures. As well, a training 
video is being prepared for retraining and updating purposes. 

#5 Technologists to review APIOS Technical and Operating Manual with 

operators, 
action The review of operating procedures in recommendation #4 
includes this review. 

#6 Detailed specifications of instrument checks. 

action Instrument checks are presently specified for all instruments. 
The complexity of an instrument determines the amount of 
detail required for the Instrument check. If more time and 
manpower become available in the programme, detailed 
instrumentation manuals will be prepared for all 
instrumentation. However, it is felt that the present 
instrument checks are more than adequate to ensure the proper 
operation of the networks. 

#7 Sangamo/MIC sensor resistance and clutch setting should be set more 

often by technologists. 
#8 During winter months, snow and ice should be cleaned off the 

instrument housing of the Sangamo/MIC and Aerochem. 
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#9 Operators should be Instructed by technologists on how to keep 

on-site log books. 
action Technologists have been instructed to carry out these 
actions. 

#10 Incorporate log book recorded information into data screening and 
validation procedures. 

action Any information recorded in the log book that is relevant to a 
sample Is presently entered onto the operator sheet under "Any 
other factors affecting sample". To attempt to incorporate 
log books on a routine basis into the data screening process 
would be very difficult and time consuming. It is felt that 
if a sample's representativeness is questionable, the log book 
can be consulted to aid in the validation procedure. 

#11 Technologists should review filter handling procedures (especially 
those that could provide a potential contamination; type of 
tweezers, washing of filter packs). 

#12 Technologists should carry out routine calibration and maintenance 
of instrumentation. 

#13 Technologists should minimize the exposure time of all precipita- 
tion samples, event and cumulative, to temperatures above 4'C. 

#14 Technologists should alert head office of site changes as described 
in the Technical and Operating Manual. 
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#16 Technologists should not park vehicles near samplers. If a vehicle 
must be brought close to the sampler location, it should never be 
left running. 

action Technologists have been instructed to carry out these 
actions. 

#15 Head office should have an upgrading training session for APIOS 

technologists. 
action As soon as technologist vacancies are filled, an upgrading 
training session will be held. 

#17 Operator Sheets should be compared with Field Sheets periodically 

by QA Coordinator for errors in data transfer. 
#18 Field Sheets should be checked periodically against database for 

errors in data transfer. 
#19 QA Coordinator should periodically verify that technologists are 

carrying out storage gauge depth determinations correctly. 
action These types of checks will be carried out on a routine basis. 

#20 Technologists should be made aware of deviations in sampling dates 

and times at their sites, 
action Technologists are aware of these deviations. Technologists 
have requested operators to follow sampling dates more 
closely. 
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#21 Quality of data collected at Gowganda, Moonbeam, Attawapiskat, Ear 
Falls, and Pickle Lake may be questionable. 

action As indicated in the actions responding to recommendation #2, a 
review of all sites was undertaken. The Milton and Attawapis- 
kat sites, based on the 1982 data, were the only sites removed 
from the networks due to evidence of local contamination. 

#22 To restructure the APIOS Technical and Operating Manual into separ- 
ate manuals in order to gear for the intended users, 
action Presently, the manual is divided into major sections, with 
each section geared to the intended user. There are no advan- 
tages in restructuring the present manual. 

#23 Preparation of the QA manual. 
action A QA manual has been prepared. 

#24 Instrumentation servicing procedures for both operators and tech- 
nologists should be prepared. 

action Operators are not required to service any instruments. If 
time and manpower becomes available, servicing procedures will 
be developed, however at present most technologists have 
sufficient experience and fundamental skills to repair and 
service monitoring equipment. 
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#25 The method currently used to fit sample bags should be discontinued 

and another method adopted. 
action The improved method, if followed correctly, will result in a 
properly placed bag in the collector. 

#26 Storage procedures for cumulative precipitation samples for - 
operators and technologists should be developed and included in the 
operating procedures. 
#27 Filter handling blank procedures should be modified to include all 

potential contamination areas, 
action Storage procedures will be prepared. Filter handling blanks 
have already been modified to include all potential contamina- 
tion areas. 

#28 A schedule detailing which samples and gauges are to be used at 

different times of the year should be included in the operator 

procedures. 

action During the Spring and Fall, technologists are instructed to 

set up both types of gauges. Operators are instructed to use 

whatever gauge is appropriate during these "transition" 

periods (i.e. use the Nipher gauge for snow events and the 

rain gauge for rain events). 

#29 Consideration should be given to changing the orifice diameter of 
the various collectors and gauges to reflect an orifice diameter 
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equal to the diameter of the container measured across the orifice 
opening from the midpoint of the gauge or bucket wall. 
action The knife edge of the M.I.C. collector does not need any 
consideration. The storage gauge measurement is a depth 
measurement, not a volume measurement and the knife edge of 
the Nipher gauge also does not require any consideration: The 
Aerochem Metric bucket edge is less than 2 nm and is not 
considered significant. 

#30 Modificatons should be made to the M.I.C. collector to improve 
reliability, operation, and performance and to allow standardiza- 
tion of instrument performance. 
action Heaters will be installed inside the collector housing to 
maintain proper temperatures for P.C. board components and 
reduce condensation. 

#31 Modifications to Aerochem Metrics Sampler. 

action This recommendation was made in a previous audit and has 
already been acted upon. 

#32 Modification of the Metrex sequential sampler. 

action Flow rate determinations are presently more than adequate 
{calibrated by mass flowmeters). Installation of mass flow- 
meters on every sampler is too expensive. Adding time coun- 
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ters on each channel to ensure clock accuracy is considered 
over-kill. We are only concerned in ensuring that reported 
volumes are accurate. 

#33 Replace low- volume sampler pump with a more reliable pump. 

action The auditor has not demonstrated that the present pump is not 

reliable. Present pumps last, on average, over 2 years. 

Considering these pumps are running outdoors 24 hrs/day every 

day, it is felt that they are reliable. 

#34 Replace the storage gauge winter solution with an immersion heater 

In water. Standardize storage gauge orifice height. 
action Storage gauges have been replaced with cumulative Nlpher 
gauges. 

#35 Reinforce Nlpher shielded gauge support stand. 

action Technologists have been Instructed to do so where required. 

#36 Provide operators with caps for the Nipher shielded gauge collec- 
tion vessel. 
action These have been provided. 

#37 All sites should have GFIC's on 115 VAC supply line. 
action Technologists have been Informed to ensure that all sites have 
a GFIC circuit (if not - a GFIC receptacle is Installed). 
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#38 On site receptacles should have a cover. 

action Technologists have been instructed to install covers at sites 
where problems have been encountered. 

#39 APIOS siting criteria should be added to in the following areas: 
no objects within 5 metres of a sampler; 
sites located in agricultural areas should have a windbreak 

for 360"; 

windbreaks on site are very desirable especially in the pre- 
vailing wind directions; 

due to their effect on airflow, buildings should be located if 
possible four to five heights away from the sampler; 
on site pollutant sources (e.g., woodburning stoves) should be 

avoided; 

windbreaks should be located in between roads and the 

sampler. 
action Recoitmendations by the auditor such as "windbreaks on site are 
very desirable especially in the prevailing wind direction" 
are indeed desirable, however if all these were to be incor- 
porated into our criteria, the criteria would be too restric- 
tive. Present criteria are major criteria to be used for the 
APIOS network. 
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#40 APIOS should develop a formalized corrective action system. 

#41 APIOS should develop a formalized training programme for new APIOS 
network personnel and also a programme to update and upgrade exist- 
ing personnel. 

action These have been Incorporated into the QA manual. 

#42 Additional manual data checking should be carried out by technolo- 
gists and the QA Coordinator as follows: 
action Most of these recommendations have been incorporated. 

#43 Upper and lower limits of Integrity checks for screening cumulative 
data are determined from historical event data. It should be veri- 
fied that these limits would not be significantly different if 
determined from cumulative historical data, 
action This verification is to be undertaken in the future. Initial- 
ly, validated event data was used since no cumulative data was 
available at that time. 

#44 Data should be screened for outliers prior to gross limit checks 

being carried out. 
action The gross limit check is an iterative procedure. The gross 
limit check is used to identify outliers. 
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#45 The groupings of sites used for the outliers tests should be 
checked to see whether any one site repeatedly experiences out- 
liers. 
action Intersite correlations are presently under review and informa- 
tion from this evaluation will be used to verify site group- 
ings. 

#46 For the outlier test, the log of parameters should be used when 

applying the Dixon Ratio Test, 
action If the distribution of the parameters shows a normal distribu- 
tion the Dixon ratio test can be used without taking the log. 

#47 Consideration should be given to reorganizing the regional techno- 
logists' responsibilities to decrease large time allocation that is 
presently necessary for travel to and from the monitoring sites. 
action This recommendation is presently being evaluated because of 
the "regional ization" of positions which has just taken 
place. 
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GUIDELINES FOR THE COMPLETION OF 

"MOE SITE DESCRIPTION QUESTIONNAIRE" 



PAGE 

# 



1-2 
2 



3 
3 
3 

3 
3 

3 
3 



SECTION 
# 



II 



II 



NAME 



Site 
Identification 



Site Details 



Site Details 



ITEM 



1-8 

9 

10-14 
15 



2 
3 

4 

5 

6 

7 
8 



GUIDELINES 



Self-explanatory 

Park, government facility, 
private property, climat site, 
institution, conservation area 

Self-explanatory 

Modified Sangamo/MIC, Aerochem 
Metrics, Nipher, Standard rain 
gauge, Storage gauge, Lov/-vol , 
Metrex sequential , 
Fisher-Porter Rain gauge 

Precipitation chemistry, 
standard precipitation depth, 
air quality, meteorological 

Forest clearing, open area, 
field 

Flat, hill, sloping, bowl 

Self-explanatory 

Grass, grain, soil, sand, moss, 
rock, gravel 

Self-explanatory 

Walk, car, 4-wheel drive, 
snowmobile, snowshoe 

Self-explanatory 

Coniferous trees, deciduous 
trees, wires, poles, fences, 
towers. Building height above 
sampler not above grade 

Distance, direction, general 
description of area as well as 
typical heights of snow or 
amount of dust 
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PAGE 

# 



5 
6 



SECTION 

# 



III 



NAME 



ITEM 



11 



12 



13 



14-17 



Site 
Influences 



GUIDELINES 



Self-explanatory 

Land use: cultivation, orchard, 
crops, pasture, forest, water, 
livestock, garden 

Ground cover: grass, shrubs, 
gravel, soil, sand, moss, rock 

Describe any physical changes 
to site or surrounding area 
anticipated, i.e., new 
buildings, new industry, new 
pollution sources, new roads, 
new road surfaces, new gardens, 
new poles or wires, cut trees 
or exposed earth, etc. 

Locked gate, guard dog, 
flooding in spring, large snow 
drifts 

Self-explanatory 



Self-explanatory 

Access roads, paved roads, 
unpaved roads, parking lots, 
garages, major highways within 
5 km 

Lakes, rivers, railroads, 
airports, canals 

Cultivation (specify crop 
type), orchards, pasture, 
grazing, trees 

Local: sewage lagoons, 
salt/sand piles, gardens, 
gravel pits, marshes, 
livestock, exposed earth, 
oil /gas wells, construction 
activity 
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PAGE 

# 



SECTION 

# 



8-9 
10-11 

12 



IV 



NAME 



ITEM 



13 



14 



sue Diagram 



Topographical 
Map 



Aerial 
Photograph 



Site 
Photograph 



7-9 

1 



GUIDELINES 



Area: Power plants, heavy 
industry, light industry, open 
pit mines, foundaries, mines, 
saw mills, pulp & paper plants 
within 50 km 

Towns within 5 km , important 
reference points, major urban 
centres within 50 km and 
special notes for large urban/ 
industrial centres (e.g., 
Toronto) 

Self-explanatory 

Use symbols as indicated in 
legend 

Attach a section of the most 
appropriate topographical map 
which shows the APIOS site and 
surrounding area 

Attach a section of the most 
appropriate aerial photograph 
which shows the APIOS site and 
surrounding area 

Include photographs from the 
four compass points with the 
sampler in the foreground 
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INFORMATION SOURCE DESCRIPTION 
FOR: MOE SITE DESCRIPTION QUESTIONNAIRE 



SECTION 


QUESTION 


SOURCE 


I SITE 


1, 2 


Operator/Regional Technician 


IDENTIFICATION 


3. 4 


Auditor 




5 


Operator/Regional Technician 




6 


MNR/Road or District Map 




7, 8 


Topographical Maps 




9 


Operator/Auditor's Observations 




10 


Operator 




11 


Regional Technician 


i 

1 ' 


12 


QA Manager/Regional Technician 




13 


Operator 




14 


Auditor/Operator 




15 


Auditor Visual Inspection 


II SITE DETAILS 


1. 2. 3, 4 


Auditor's Observations 




5, 6. 7 


Operator 




8 


Auditor's Observations 




9, 10 


Operator 




11 


Aerial Photograph 




12 


Operator 




13 


Operator/Auditor's Observations 




14 


Operator 




15 


Operator/Auditor's Observations 




16 


Auditor to measure with tape 


: 1 


17 


Operator 


III SITE 


1. On Site 


Auditor's observation - to be 


INFLUENCES 


Observation 


measured with a tape 




1. Local 


Operator 




Area Sources 






2, 3, 4, 5, 
6 


Operator 

1 
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SECTION 



INFORMATION SOURCE DESCRIPTION 
FOR: MOE SITE DESCRIPTION QUESTIONNAIRE 



III SITE 

INFLUENCES 
(cont'd) 



QUESTION 



SOURCE 



8,9 



Auditor observation and 
information obtained from 
topographical map 
Operator's Observation 
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MOE SITE DESCRIPTION AUDIT QUESTIONAIRE 



I SITE IDENTIFICATION 



1. Station Name: 

2. Station Number (SIS)^ 

3. Date of Visit: ^Time: 

4. Auditor: 



5. MOE Region 

6. County: Townwship:_ 

7. Latitude ' ' " Longitude: 



UTM Co-ordinates 



8. Elevation m 

9. Type of Site (park, gov't facility, private property, etc): 



10. Site Address 



Mailing Address 
(if different 
from above) 



11. Brief description of how to reach site 



12. Name of Primary Operator 

Alternate Operator 
Regional Technician 

13. Operator*s Address 



Mailing Address 

(if different 

from above) 

Telephone Number Home ( ) Business ] )_ 



Alternate Operator Home ( ) Business ( ) 
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MOE SITE DESCRIPTION AUDIT QUESTIONAIRE (Cont'd) 



14. Sample Types Collected at Site: 

Cumulative Wet Cumulative Dry 

Event Wet Event Dry 

Other (describe) Special Study_ 

15. List monitoring instrumentation on site. 

Instrument Measurement 

Type/Make 
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II SITE DETAILS 



1. Physical characteristics of site (i.e. clearing, open area) 



2. Site Topography (e.g. flat, hilly, sloping): 



3. Ground slope at site:_ 

4. Ground cover at site: 



5. Access to site Suimner: Good Fair Poor N/A_ 

Winter: Good Fair Poor N/A_ 

6. Method of access: Summer: ^ 



Winter: 



7. How close do vehicles normally approach sampler? 



8. Presence of windbreaks within 100m of sampler: 

Type Height (m) Distance (m) Direction 



9. Where do snow and dust drift and accumulate?; 



10. Prevailing wind direction during precipitation?: 

W i nter : 

Summer : 
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II SITE DETAILS (continued) 



U. Land Use: 




•Urn 



"^•ItMM 





Land Use 


% Ground Cover 


1. 






2, 






3. 






4. 






5. 






6. 






7. 






8. 







12, 



13. 



Future land use, planned or expected (construction, new 
cultivation, clearing, etc): ^^_ 



Are there any logistical problems which prevent the sampling 
site from being easily approached or operated (locked gate, 
guard dog, flooding in spring, large snow drifts): 



14. 



15. 



What is the electrical power available at the site? 

(volts) (Amps) .(# of circuits) 



Is the circuit on a: 
Receptacle GFIC 



No GFIC 



Circuit Breaker GFIC 



Battery 



16. 
17. 



Distance from collector receptacle to circuit panel; m 

What are the number of power failures: 

per month per year __^________ 
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III SITE INFLUENCES 

1. General potential sources of contamination/interferences 



Type 


HEIGHTdn)/ 
SIZE 


DISTANCE 
(m) 


DIRECTION 


On-Site Obstructions 








Trees 


Poles 


' 






Wires 








Buildings 




; 




Other - 








Local Area Sources (within 5Km) 








Sewage Lagoons 


Salt/Sand Piles 








Gardens 








Gravel Pits 








Marshes 








Livestock 








Exposed Earth 








Oil /Gas Wells 


1 






Construction 








Human Activity 






' 
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III SITE INFLUENCES (continued) 



2. Vehicle Related 



TYPE 


NAME 


SURFACE/ 
USAGE 


DISTANCE(ni)/ 
DIRECTION 


SNOW/DUST 
CONTROL 


Highways 










Paved Roads 










Unpaved Roads 










Access Roads 
(Driveways) 








1 ' 


Parking Lots 










Garages 











3. Other Transportation-Related Influences 



TYPE 


NAME 


DISTANCE (km)/ 
DIRECTION 


USAGE 


COMMENTS 


Lake 










River 










Rail 










Airport 








, 
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III SITE INFLUENCES (continued) 
4. Agricultural Activities 



TYPE 



AMOUNT 



DISTANCE(km)/ 
DIRECTION 



HERBICIDES/ 
PESTICIDES 



5. Local and Area Pollutions Sources 



TYPE 



NAME 



01 STANCE (km)/ 
DIRECTION 



CAPACITY/ 
PRODUCT 



EMISSION 
TYPE/RATE 



("oncntxi X UMiIilK ( i m;'k^i,'.I:( <\^ 



AI.13 



III SITE INFLUENCES (continued) 



6. Population Centres 



NAME 



DISTANCE{km) 



DIRECTION 



POPULATION 



7. 



Is the site representative of the topography and vegatative 
ground cover within the surrounding area? (10 - 100 km) 



8. 



List deviations from siting criteria and all advantages and 
disadvantages of site: 



I 
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III SITE INFLUENCES (continued) 

9. Auditor's comments with respect to site (Is it a excellent, 
good, fair, poor unacceptable site?): 



(xjnrord NJcntiiu i . m'i ■■' r.iUi « 



i I — I — I — I 

Scale 




AI.16 - 



N 



Cone (11(1 S< itMititi< ('iiifxji.ii'mii 



- AI.17 - 



LEGEND OF SYMBOLS FOR USE IN SITE DRAWING 




Building 



Primary Sampler 



Aerial Cable 



-» Prevailing Wind Direction 



D Low Vol 
O storage Gauge 
A Nipher Guage 
Roads 



"[I I I |"T Railroad tracks 



M 



Poles 



North 



■* Jt Ik Fence 





\/ 
-rrrrrr 



Brush 



Coniferous Trees 



Deciduous Trees 



Sloping Ground 



H- Height 



@ Stevenson Screen 
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Site Diagram 
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Topographical Map 




LoihokI y i»."nliiii. I .,|K_i!,ii' 



AI.21 - 



Site Photographs 




I 
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Aerial Photograph 




I 
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TYPE OF MONITORING - CUMULATIVE WET 

INSTRUMENT - MIC Collector (Formerly Sangamo Type A - Modified) 

Configuration Checks (Record results on "MlC/Sangamo Type A Data 
Sheets^^l 

1. Ensure gasket on cover contacts knife edge all the way around, 
especially on farthest position of sampler cover. 

2. Ensure sample bag fits container well. 

3. Only galvanized support should be used. 

4. Original steel U-bolts were to be replaced with stainless steel U- 
bolts. 

5. Knife edge should be level across opening. Qualitatively test by 
placing clean plastic over open wet bucket with a level on top. 
Note whether it is level, slightly out of level or not level. 

6. Check that the underside of sensor grids is not blackened or dis- 
coloured. 

7. Check that both grids are warm, not hot to the touch. 



Calibration Checks (Record results on "MIC/Sangamo Type A Data 
SheetT^) 

1. Instrument Response - Using CSC Ohm Ranger find the resistance of 
the precipitation sensor necessary to activate the sampler. A 
resistance of 220 Kn should cause the sampler to activate. (Figure 
1). 
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2. Heating of Sensor - Measuring the voltage across the two heating 
terminals on the sensor a pulsing ACV of 5 volts should be detec- 
ted. If the voltage is not pulsing there is an instrument problem. 
Test the heater by placing voltmeter probes across heater pins 
(Figure 2). The grids should feel warm to touch not hot or cold. 

3. Clutch Adjustment - Activate and stop sampler midway by turning on 
and off switch. Place the hook of a spring gauge at the junction 
of a sampler arm and cover. Holding the other end of the gauge 
(ring) turn on sanpler. The arm should start to move until the 
gauge reads 14 lbs at which point the instrument clutch should 
start to slip. Measure the clutch slip. (Figure 3). 

4. Limit Switch Adjustment - Activate sampler and when sampler cover 
reaches midway turn off sampler and unplug from receptacle. Remove 
instrument screen and cover. 

a) Depress limit switch plungers (2x) until they click and then 
release. Switch plungers should spring back without sticking. 

b) Visually inspect switch plungers. Note any deformity of 
plunger. 

Plug sanpler back into receptacle and turn sampler on observing 
angle of adjustment screw and limit switch plunger. 

c) Note if adjustment screw is contacting plunger on such an 
angle as to damage switch. 

d) Note if adjustment screw at end of travel is bending limit 
switch support bucket. (Figure 4). 

5. Gasket Seal on Knife Edge - Place a thin layer of clear compres- 
sable foam on the edge and allow sampler cover to move and compress 
foan will allow a check of gasket sealing ability. After a short 
interval activate sampler and inspect knife edge impression on 
foam. (Figure 5). 

6. Sampler Delay Time - Short sensor with metal strip or moistened 
finger and start stopwatch. Stop stopwatch when sample cover 
starts to return to wet side of instrument. 
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Heating 
Circuit 



resistance placed 
between these two 
points 



Thermistor 
Temp. Sensing Circuit 




Sensor grid 



FIGURE 2 



Sensor grids 




To Sangamo 



Test heater by placing 
voltmeter probes across 
heater pins 



Sensor Support 



FIGURE 3 



MIC/ 
Sangamo 

body 




hold ring tight 



gyage should 
read 141 bs before 
clutch slips 
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FIGURE 4 



plunger 



limit_ 
Switch 



support 



Bracket 




support bracket 



adjustable 
screw 



foam 



knife edge 



bucket 




sampler cover 
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MIC/MODIFIED SAN6AM0 TYPE A DATA SHEET 



Site: 



Serial #:_ 
On Stand? 



Model #: 



Height above ground_ 





N - S 


E - W 




WET 


WET 


(Bucket) SAMPLE ORIFICE (cm) 






(Bag) SAMPLE ORIFICE (cm) 







Level 



Precipitation Sensor Checks 



Knife edge gap 

Instrument Response 

Heating of Sensor 

Warm to touch 



mm Delay Time 



kn Clutch Adjustment 



volts pulsing? 



sec 

lbs 



Blackened/Discolored? 



Comment on: 



Limit Switch Adjustment; 



Gasket Seal on Knife Edge: 



Hood Movement: 



Bag Fit: 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC 



OPERATOR: 
SITE: 



Operator's Duties 

1. What time does operator change sample bags? If he 
can't change at the proper time, what does he do? 
(phone technican, change bag when he can, contact 
alternate operator). . 



Yes 



2. 



Does the operator, while removing the sample bag from 
the sample bucket, allow for any chance of 
contamination of the precipitation sample? Several 
points that may present contamination are: 

is sample bucket removed from sampler prior to 

changing sample bag? 

does the operator wear clean, new gloves to 

remove the knife edge? 

does he handle only the outside of bucket and 

knife edge? 

is he careful not to contaminate knife edge when 

its taken off? 

is bag removed in such a way as to not introduce 

contamination from top and outside portions of 

bag? 

is operator careful not to stand directly over 

pail? 

is there a potential for contamination by hair, 

coat, etc.? 



Comments: 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 

3. Are the sample bags properly handled to prevent 

damage, sample loss and allow for proper handling by 
laboratory staff? Several points should be checked: 
is air forced out of bag before sealing? 
is bag tied near enough to the top so that it 
can be leached for metals analysis, but far 
enough down so that none of the contaminated 
surfaces are exposed to sample? How is bag tied 
and sealed? _^ 



Comnents; 



does operator use a container to store and 
transport sample? 



are bags promptly labelled (What information is 
recorded on label}? 



for winter operation with long tube how is bag 
removed? 



where does operator store sample for pick up by 
technician? (winter and suntner). 



how long does it take the technician to come and 
pick up samples? 



Yes 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 

4. Are the sampler components carefully cleaned? 
Several points to be checked: 

does operator clean the following before put- 
ting in new bag: 
Sensor grids with HgO and ETOH? 
Hood gasket with H2O and ETOH? 
does he use new gloves and Kimwipes for above? 



what is the cleaning procedure under cold 
conditions? __^ 



Comments: 



5. What problems does the operator experience with the 
sampler? How does he handle them? Some things that 
can happen: 

does wet sample bag get stuck in container when 
being removed? 



Yes 



what does operator do if this happens? 



No 



what procedure is followed if the bag comes out 
damaged? 
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CUMULATIVE PRECIPITATION AUDIT - SAN6AM0/MIC (continued) 



Operator's Duties 



how often are leaks observed in bags? 



what does he think causes it? 



Comments: 



6. In placing a new sample bag in the collector, does 
the operator observe the following points: 

are new gloves worn when touching inside of 

bag? 
- is operator careful not to touch inside of bag 
any more than is necessary? 
1s bag forced to bottom of container by filling 
with air? 

is operator careful not to put hand inside bag 

when installing it? 

is enough bag left over the edge to permit knife 

edge to hold bag? 

are new gloves used to flatten bag against the 

pall? 

Is collar cleaned, rinsed and dried before 

replacing on container? 



Yes 



No 
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CUMULATIVE PRECIPITATION AUDIT - SAN6AM0/MIC (continued) 



Operator's Duties 



Does the end result produce a well fitting bag? 
(smooth surface, no overhang). 



Comments; 



Does the operator have a good knowledge of what he is 
expected to do? Specific points that the operator 
should be asked: 

does operator have written instructions for 

procedures? 

which procedures does he have? 



does he think he was adequately trained? 



when was the last time the regional technician 
reviewed procedures with him? 



does he have any particular problems with the 
sampling? 



Yes 



No 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 




Comments; 



8. Does the operator have sufficient supplies on site? 
He should be asked: 

does he receive regular shipment of supplies or 
as needed? . 



does he have enough of the following: 

sample forms? 

bags? 

DIHpO and ethanol? 

kimwipes? 

gloves? 

has he ever run out of supplies? 



Comments; 



9. What comments does the operator have about the 
programme? 

does the operator understand why he is 
collecting the samples?^ 
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CUMULATIVE PRECIPITATION AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 



does operator understand why all the special 
precautions (glove, cleaning bags etc.) are 
required? ^ 



does operator get any feedback on prograrmne? 
Would he like feedback? How often? 



Yes 



No 



does he have any special comments with regards 
to the sampling, site, technician or programme 
as a whole? 



does he feel the pay is adequate? 



Comments: 



10. Does the operator check sampler performance on a 
regular basis? 

does he check that hood seals tightly 
does he check that the sensor grid is warm? 
what other instrument checks does he perform 
periodically? __, 



how often does he check the instrument? 
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CUMULATIVE PRECIPITATIOM AUDIT - SANGAMO/MIC (continued) 



Operator's Duties 



Comments: 



U. Does the operator maintain a site log book? 
what type of information is recorded _ 



how often is information recorded? 



Comments: 



Yes 



12. Are sample submission sheets correctly and completely 
filled out? 



13. 



Does the Regional Technician visit the site? 
how often? 



No 



does he talk to the operator on each visit, or 
is the sample just left at a location for 

pi ckup? ., 
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CUMULATIVE PRECIPITATION AUDIT - SAN6AM0/MIC (continued) 



Operator's Duties 



do the technician and operator discuss sample 

problems? 

other than talk to the operator what else does 

the technician do on his site visits? 



Comments: 



Yes 



No 
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TYPE OF MONITORING - CUMULATIVE WET 

INSTRUMENT - STORAGE GAUGE 

Configuration Checks (Record results on "Storage Gauge Data 
Sheet") 

1. Storage gauge opening should be level. Qualitatively test by 
placing level across opening. Note whether it is level, slightly 
out of level or not level. 

2. Measure the height of the bottom of the storage gauge from the 
ground. 

3. Measure the height of the orifice of the storage gauge from the 
ground. 

4. Visually note any floating debris in guage or adhered to Inside 
gauge walls. Using metre stick check for any sunken debris. 

Calibration Checks (Record results on "Storage Gauge Data Sheet") 

1, Gauge Opening - Measure inside opening of storage gauge with a 
ruler. Repeat at 90' from first reading. 

2. Solution and Capping Layer - Visually note whether solution is 
methanol /ethylene glycol (slight alcohol - like odor) or water (no 
odor). 
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STORAGE GAUGE DATA SHEET 



Site: 



Serial #: Model #: 



On Stand? Storage Gauge Height above ground_ 

Orifice Height above ground 



How Is the base of storage gauge supported? 



Sample Orifice (cm) N-S level N-S 

E-W E-W 



Solution Type: Methanol Ethylene Glycol 
Water 
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CUMULATIVE PRECIPITATION AUDIT - STORAGE GAUGE 



OPERATOR: 
SITE: 



Operator's Duties 



1. Does operator take a storage gauge measurement? 



2. How does the operator take the depth measurement? 



Comments: 



3. Does the operator have a thermometer present for 
winter time measurements? 



Yes 


No 
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TYPE OF MONITOIRNG - CUMULATIVE DRY 



INSTRUMENT - METREX LOW-VOL SAWLER 

Configuration Checks (Record results on "Low-Vol Sampler Data 

Sheet") ' ^^^^ ^ ^Flexible Tubing 

(Tygon or Nylon) 
Figure 1: 



Environmental 
Housing 

Low Vol - 

sampler 

Rotameter 




a 



Filter Pack Plate 



Filter Pack 

— Shelter 

Support Stand 



ground level 



1. Note any deviations of the sampling configuration, (see Figure 1). 

2. Observe flow rotameter. Is there a black ring on walls of rota- 
meter? If yes describe ring thickness, colour (dark black, grey, 
brown). 

3. The filter pack should be level. Qualitatively check by placing 
level across filter pack case. Note whether it is level, slightly 
out of level or not level. 

Calibration Checks 

1. Flow - Calibration of Rotameter ; Turn off Lo-Vol Sampler and dis- 
connect suction line from the back of the sampler. Place a 0-5 
1/min mass flowmeter in line. Turn sampler and mass flowmeter on 
and allow 20 minutes for warm-up. Check for leaks 1n system. The 
output of the mass flow-meter should be connected to a digital 
voltmeter (scale 0-5 VDC). Use the flow control adjust on the 
sampler to perform a multi-point (10 values) calibration allowing 
30 seconds per reading to allow the flow rate to stabilize (record 
results on "Rotameter Calibration Table"). The calibration should 
cover a range of 0-5 1/min, 
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2. Volume - Calibration of Volumetric Counter (Gas meter) The instru- 
ment set up for calibration Is the same as in the "Calibration of 
Rotometer" except that once the flow controller has been set to 
approx. 2 1/min it should not be adjusted. When unit on volumetric 
counter changes start stop watch and record volumetric counter 
reading and mass flowmeter reading. Each time the volumetric 
counter changes (approximately 5 minutes) record volumetric counter 
reading, mass flowmeter reading in VDC, Low-Vol rotameter reading, 
and time (record results in "Volume Counter Calibration Table"). 



Coiifortl Scicnlitk ("(ii|xtrjlion 



AI.41 - 



METREX LQW-VOL SAMPLER DATA SHEET 



sue: 



Serial #: Mo<*ei *•- 



On Stand? Height above ground^ 



E-W 



Sample Orifice (cm) N-S level N-S 

E-W 

Sampling line and tower in good condition? 

Kinks in suction line?^ 



Tower firmly supported, straight and steady? 
Low-Vol sampler - covered? ^ . 



located so as not to be influenced by other nearby 
samplers or objects? . 



Rotameter Condition - grey, brown or black ring? 
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Rntaneter Calibration 


Temperature 


















Reading # 


Low-Vol 

Rotameter 

Reading 

1/min 


Voltmeter 

Reading 

(VDC) 


Mass Flowmeter 
Reading 
(1/min) 






1 


, 


1 








2 












3 












4 












5 












6 






I 






7 












8 












9 












10 










Volui 


«> Counter Calibration 


Temper at 


ure 












Reading # 


Time 
(min & sec.) 

II 


Volume 

Reading 

{1 X 10) 


Voltmeter 

Reading 

(VDC) 


Mass Flowmeter 
Reading 
(l/m1n) 


Low- Vol 
Rotometer 
Reading 
(1/min) 


1 












2 












3 












4 li 












5 












6 












7* 






'' 






8 












9 


1 










10 
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CUMULATIVE DRY DEPOSITION - LOW VOL 



OPERATOR: 
SITE: 



Operator's Duties 

1. Describe the procedure used by the operator in 
handling filter packs. Does he do the following? 
handle filter packs only by using a plastic bag 
or on the filter pack? 



where are the filter packs stored? 



Yes 



No 



Comments: 



2. What information is recorded on data sheets? 
flow rate - start and finish? 
volumetric reading? 
time of filter pack changeover? 
operator comments? 



Comments: 



3. What training did operator receive on the sampler 
handling filter packs? 



has he been retrained since? 

does he feel training was adequate? 



Comments: 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER 



OPERATOR ; 
SITE: 



Technician/Operator's Duties 

1. Describe operators procedures prior to lowering the 
boom? Does he do the following? 

field sheet prepared for each filter pack or 

tower? 

are sampling channel flows transferred to data 

sheet? 

is the time showing on mechanical clock correct? 

1s the flow rate checked? 

Is the sampler turned off before the boom is 

lowered? 



Comments; 



2. After the boom has been lowered, describe the pro- 
cedure used to reload filter pack. Is the following 

done: 

Is a new clean sample bag used to remove filter 

pack? 

what type of bag is used? 



are either the filter packs or bags labelled 
corresponding to sanple channel? If not how are 
the filter packs Identified? 



Is the filter pack sampling for that day 
transferred to another sampling channel? What 
channel ? — 



Yes 



No 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Technician/Operator's Duties 

Yes No 

is the sampler turned off while filter packs are 

being removed? 



Comments: 



3. 



Describe procedure used to reload filter packs. Is 

the following done: 

is the filter pack installed on tower while 
still in bag or is it removed by hand and then 
installed? 



when are the sampling channels 2eroed7_ 



is the sampler restarted when the boom is on the 
ground or after being raised?^ 



does the technician cycle through all channels 
after sampler is restarted? 



Yes 



4. What information is recorded on the sample sheets and 
In the site log book? 



5. How frequently is the sampler checked, daily, weekly? 
Who does the checking the technician or someone else? 



No 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Technician/Operator's Duties 

6. At the laboratory how long do the filter packs usual- 
ly sit before being unloaded? . 



7. 



8. 



Does the technician unload filter packs in a: 
laminar flow chamber? 
fume hood? 
office? 
laboratory? 

Are there any exposed contaminants in filter proces- 
sing area? 

anyone smoking nearby ?_ 




any exposed chemicals - especially H2SO4 or HNO3 



is the area generally clean or dirty?_ 



Comments: 



9. What kind of work surface does the technician use? 

teflon coated table top? 

lab bench? 

lab bench with plastic on top? 

office desk? 
Comments: . — 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Technican/Operator's Duties 

10. Is the technician's system for labelling bags easily 
understandable and logical? (describe the system 
used) 

is it done all at once prior to unloading any 
filter pack?^ 



is it done as filter packs are unloaded?_ 



Comments: 



11. Describe the procedure used for handling the 
filters? Are the following points observed? 

are teflon tweezers used for filter handling? 

are tweezers stored in methanol prior to use? 

are they wiped with new Kimwipe before touching 

filters? 

are separate tweezers used for impregnated and 

unimpregnated filters? 

are Nylon and Impregnated W41 filters folded 

twice before placement in sanple bags? 

are gloves worn during filter pack unloading? 



Comments: 



12. Describe the procedure used for the washing and 
drying of filters? Is the following done: 

where are filter samples stored after unloading 
from filter packs? 



Yes 



No 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Technican/Operator's Duties 



are filter packs completely disassembled prior 
to washing? 

are the filter packs rinsed after washing? What 
rinses are used? 



how many of each type of rinse are performed?_ 



Yes 



how are the filter packs dried? If air dried, 
are they covered during drying? What covering 
is used? 



if dried in an oven, how long, what oven 
temperature? 



are the filter packs used after drying? If not, 
how are they stored? 



No 



Comments: 



13. Describe the procedure used for reloading of filter 
packs. Is the following done: 

does the technician clean off the area where 
filter packs will be loaded? 
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EVENT DRY DEPOSITION - SEQUENTIAL SAMPLER (CONT'D) 



Technican/Operator's Duties 



Is the work surface a lab bench, Teflon overlay 
board, "Kimwipes" over lab bench or other area? 



are cleaned tweezers kept in separate breakers 

one for un impregnated and one for impregnated 

filters? 

when removing filters from bag does technician 

push his hand into bag or only tweezers? 



when placing collar on filter pack is technician 

careful not to get hand near open exposed face 

of filter? 

is the inlet of the filter pack checked to 

ensure the 0-ring has not failed? 

is the loaded filter pack numbered and stored in 

a new "Zip-lock" or "Whirl-pak" bag? 



how are loaded and bagged filter packs stored 
prior to shipment to field?_ 



Comments: 



14. Are any of the filters from Toronto ever damaged? 
(i.e. nicks out of the sides). If so what is done 
with them? 



Yes 



No 
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TYPE OF MONITORING - EVENT WET 

INSTRUMENT - AEROCHEM METRIC 

Configuration Checks (Record results on "Aerochem Metric Data 
Sheet") 

1. The Aerochem Metric has a reputation for being blown over in winds. 
The sampler should be securily mounted by either stakes, wire or 
cross pieces. Insure that the sampler can not be pushed over. 
(Figure I). 

2. The instrument should be level. Qualitatively check by placing 
clean plastic over the bucket with a level on top. Note whether it 
is level, slightly out of level or not level. 

3. Check cleanliness of a) gasket, b) sensor, c) sensor body. 
Calibration Checks (Record results on "Aerochem Metric Data 



Sheet" ) 

1. Heating of Sensor - When sampler is not activated sensor should be 
cold to touch. When activated, the sensor should get hot after 
approximately 5 minute delay, and stay hot until activating precip- 
itation evaporates. 

2. Cover Seal Integrity - A thorough visual inspection of cover check- 
ing for: a) holes in polyethylene gasket cover; 

b) signs of gasket foam crumbling; 

c) moisture on inside surface of polyethylene. 
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FIGURE 1 




Ground Cover 



Stakes In Ground 
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Site: 
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AEROCHEM METRIC DATA SHEET 



Serial #: 



Model #: 



Sample Orifice (m) N-S 

E-W 



level N-S 
E-W 



Secured by: stakes 
wire 



cross pieces 



sturdy? 



Cleanliness - Gasket 

- Sensor 

- Sensor Body 
Any holes in gasket cover? 
Is gasket foam crumbling? 

Any moisture on inside surface of cover 
Sensor heating properly? 



Yes 


No 



































Comments: 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC 



OPERATOR: 

SITE: 



Operator's Duties 

1. What time does the operator check the sample bag for 
precipitation? If he cannot change at the proper 
time, what does he do? (phone technician, change bag 
when he can, contact alternate operator). _ 



2. 



Does the operator, while removing the sample bag 
from the sample bucket, allow for any chance of 
contamination of the precipitation sample? Several 
points that could lead to contamination are: 

is the sample bucket removed from the sampler 

prior to changing sample bag? 

does the operator wear clean (new) plastic 

gloves to handle the bag? 

is the outside of the bag folded up allowing 

contamination to enter bucket? 

is the operator careful to not stand directly 

over sample bucket while removing sample bag? 

Is there potential for contamination from 

operator's hair, clothes etc.? 



Comments: 



Yes 



No 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 



Operator's Duties 

3. After the bag is removed from the sample container, 
is air evacuated from the bag after being 
removed from container? 
is the top of the sample bag tied? 
how far from the top of bag is it tied? 
what is the sample bag tied with? 
any chance for othe contamination (clothing, 

etc)? 



Yes 



No 



Comments: 



4, For reloading the sample bucket: 

in placing new bag in sample container, does 
operator wear gloves? 

does the operator touch the inside of bag when 
loading into sample container with only new, 

clean gloves? 

any chance for other contamination (clothing, 

etc.)? 



Comments: 



5. When 



the new bag is loaded in the sample bucket: 

does the bag fit the container well? 

how much of a decrease in orifice size results 
from poor bag fit? How niuch is the sample area 
reduced {%)? . 



Comments: 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 



Operator's Duties 

6. Describe how the operator transfers the precipitation 
sample from the sample bag to the sample bottle? 
does the operator inspect sample bottle for 
flaws? 

does he swirl bag and contents around prior to 
decanting? 

how is sample melted if partially or completely 
frozen? 



does operator pour sample directly from bag into 

sample bottle? 

does he use a clean funnel? 

does he elevate one bottom corner, clean, and 

cut it? 

describe cleaning procedure of bag and scissors? 



if sample volume exceeds bottle contents, is 
excess poured into graduate and measured? 
where is the sample transfer area? 



is a clean sample preperation area used? 



Comments: 



Yes 



No 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 



Operator's Duties 

7. Sample bottles should be labelled to correspond with 
date submission sheets: 

what Information is placed on sample label? 



is label placed on cap or bottle? 



where are samples stored for pickup by 
technician? _^^_^__ 



Yes 




how long are samples usually stored prior to 
p1ck-up by technician? 



Comments: 



8. What information is entered on the data sheets? 

is any information not filled out or incorrectly 
filled out? 



9. Is there a log book on site? 

what information is recorded? 



how often 1s Information entered Into log book? 
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EVENT PRECIPITATION AUDIT - AEROCHEM METRIC (continued) 

[ 
Operator's Duties 



10. When was the last time technician reviewed sampling 
procedures with operator? 



how often are procedures reviewed?_ 



Yes 



No 



does the operator think procedures should be 
reviewed more often? 



does he think he was adequately trained 

does he have any particular problems with the 

sampling 



Comments: 



11. 



Does the operator have sufficient supplies to last 
for one week, one month, six months? Has the 
operator ever run out of supplies? __ 



12. What comments does the operator have about the 

programme? 

does the operator undertand why he is collecting 



the sample? 



does he understand why all the special 
precautions (gloves, cleaning, etc.) are 

required? 

does operator get any feedback on programme? 

Would he like some? How often? 



does he feel the pay is adequate? 
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nPE OF MONITORING - EVENT WET 
INSTRUMENT - STANDARD RAIN GAUGE 

CONF I G URATION CHECKS 

(Record results on "Standard Ra1n Gauge" Data Sheet) 

1. Measure from collector rim to ground. 

2. Measure from top of grass to collector rim. 

3. Inspect funnel and graduate for cracks and cleanliness. 

4. Determine height of and distance to nearest obstruction, including 
sampler. 

5. Insure that gauge is emptied each morning. 

5. The instrument should be level. Qualitative check by placing level 
over gauge orifice. Note whether it is level, slightly out of 
level or not level. 

7. Pour pre-measured 50 ml aliquot into gauge and record reading. 
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STANDARD RAIN GAUGE DATA SHEET 



Site: 



Serial # 



Model # 



Conector rim to ground 

Collector rim to grass 

Graduate (Yes, No) 
Funnel (Yes, No) 



Cracked 



Clean 



Obstruction 


Height 
of Obstruction 


Distance 
from Gauge 
(m) 


1 

1 1 

1 
1 


) 





Gauge Level 



Gauge Reading 



(mm) 



I 
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EVENT PRECIPITATION - STANDARD RAIN GAUGE 



OPERATOR: 
SITE: 



Operator's Duties 

1. Is the standard gauge checked each day? 

2. How does the operator handle the following 
situation: 

if water level (meniscus) falls between two 
graduated marks? 



what is recorded if level is below 0.2 on scale? 



if there 1s water that has overflowed the 
graduated cylinder, what does the operator do? 



what is done for freezing rain? 



Yes 


No 
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TABLE 3.1 



Sites Visited During the Phase II Site Audits 



Site 



IZ£i 



Merlin 

Port Stanley 

Pickle Lake 

Forbes Township 

Quetico Centre 

Ear Falls 

Experimental Lakes Area 

Dorset 

Whitney 
Coldwater 
Gowganda 
Winisk 
Whitman Creek 



Cumulative Wet 

Cumulative Wet & Dry 

Cumulative Wet & Dry 

Event Wet 

Cumulative Wet and Event Wet 

Cumulative Wet & Dry 

Cumulative Wet 

Cumulative Wet & Dry, Event Wet 

& Dry 

Cumulative Wet 

Cumulative Wet 

Cumulative Wet & Dry 

Cumulative Wet 

Event Wet 



I 
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MERLIN 

Site Evaluation 

Advantaqes: 

representative of topography and qround cover 

no on-site obstructions 
representative of area 
good access year round 
no major industrial sources 

Disadvantages: 

incomplete windbreak 
adjacent to a barn 

Instrument Evaluation 

Sangamo Type A 

low resistance on sensor 
• instrument in good working order 

Storage Gauge 

orifice slightly out of round 
slightly off level 
well supported 

Corrective Action Required 



none 
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PORT STANLEY 

Site Evaluation 

Advantages: 

representative of topography and ground cover 
no on-site obstructions 
representative of area 
good access year round 
no urban influences 

Disadvantages: 

incomplete windbreak 
proximity to treatment plant 

Instrument Evaluation 

Primary MIC Type A 

low resistnace on sensor 
good working order otherv;ise 

Secondary MIC Type A 

low resistance on sensor 
hood movement not smooth 
good working order otherwise 

Primary Storage Gauge 

well supported 

level 

gauge orifice out of round 

Secondary Storage Gauge 
well supported . 
level 
gauge orifice out of round 
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Primary Low Volume 

1 guy wire on tower broken 
good working order 
good she! ter 
well supported 

Secondary Low Volume 

good working order 
good shelter 
well supported 

Operator Evaluation 

Operator: John ApfeVbeck 

uses only 1 pair of gloves to install and remove bags 
operator requires upgrading on sample handling techniques 
good attitude, very conscientious 

Corrective Action Required 

retrain operator 
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PICKLE LAKE 

Site Evaluation 

Advantages: 

representative of ground cover and topography 

representative of area 

CO- located at Cllmat and CAPMoN sites 

no local or area sources 

no agricultural or large urban influences 

minimal vehicular influences 



I Disadvantages: 

on site obstructions - trees 20m E/SE 



- tower 21m NE 
no windbreak to the west towards the lake 



Instrument Evaluation 
MIC 



one grid blackened 
I - clutch adjustment too strong 

overall performance good 



Storage Gauge 

well supported on wooded stand 
level 



Low Volume 

good working order 
I - supported on a good stand and in a wooden enclosure 
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Operator Evaluation 

Temporary Operator: Ron MarsH 

follows outlined sample handling procedures 
checks sampler daily 

Corrective Action Required 

remove trees 

perform minor maintenance on MIC 

train new operator 
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FORBES TOWNSHIP 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

no on-site obstructions 

well removed from urban and agricultural influences 

good access year round 

representative of local area 

complete windbreak 

Disadvantages: 

proximity to power plant 

Instrument Evaluation 

Aerochem 
Primary 

good working condition 

dry bucket in place, possible source of contamination 

Secondary 

good working condition 

dry bucket in place, possible source of contamination 

Standard Rain Gauge 

Primary & Secondary - gauges level, clean and no cracks in 
graduate 
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Operator Evaluation 

Operator: Mary Ingles 

follows all sample handling procedures outlined 1n APIOS 

Technical and Operating Manual 

excellent attitude 

very conscientious in sample handling 

Corrective Action 

remove dry buckets from Aerochems 
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QUETICO CENTER 

Site Evaluation 

Advantages: 

no on-site obstructions 

good ground cover 

reliable power source 

easily accessible 

windbreak complete except to S-SE 

representative topography and ground cover 

site representative of area 

Disadvantages: 

proximity to road 

Instrument Evaluation 

MIC - Sangamo 

one sensor blackened, hot to touch 

instrument supported well and on a good stand 

sampler activated with 200 Q resistance 

sampler generally in good working order except for 

blackened sensor grid and low resistance on sensor 

Aerochem 

sampler in good working order 

water dripping from sensor head 

dry bucket in place, possible source of contamination 

poor seal on bucket 

new motor and sensor installed 

Standard Rain Gauge 

gauge level, clean and no cracks in graduate 
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Operator Evaluation 
Operator: Austin Donobue 

- operator leans over sanple bag when changing bag 
a1r not forced out of bag before sealing 
gloves not worn to clean gasket, hand touches gasket 
same gloves used to handle outside of bucket and install 
new bag 
the instrument body and component parts should be cleaned 

more often 

operator has a good attitude and would be an excellent 

operator if retrained 

Corrective Action Required 

sensor grids on MIC to be changed 

dry bucket should be removed from Aerochem 

Aerochem should be adjusted to provide a good seal on wet 

bucket 

operator to be retained and periodically checked to ensure 

that he is following proper sample handling procedures 
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EAR FALLS 

Site Evaluation 



I 
I 
I 
I 
I 

Advantages: 
■ - representative of topography 

good accessibility and security 

Disadvantages: 

poor ground cover - gravel 

on-site obstructions 2 trees 10 m NE (~ 8 m tall) 

gasoline tank 8 m NE 

proximity to parking lot or roads 

adjacent to storage area and construction (garage) 

ground slopes E-SE 

light industry 3 km W 

Instrument Evaluation 

MIC 

limit switch needs to be replaced and set properly 

chain needs to be tightened 

no other problems with instrument 

supported well and level 

Storage Gauge 

supported on a wooden stand 
level 

Low Volume 

not operating at time of audit 

well supported inside a housing unit 
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Operator Eval^uation 

Operator: Harold Atkinson 

operator not present at time of audit 

Corrective Action Required 

site has many deviations, some of which can be ameliorated 

(e.g. cut trees, plant grass, clean-up area, etc.) 

deviations which cannot be improved include proximity to 

open ore pit mine, vehicular influence, light industries 

and poor ground slope 

relocate site, if not possible, then historical data 

should be reviewed to determine if valid samples are being 

collected. Site should be removed from the APIOS network 

if data indicates that invalid samples are being 

collected 

perform maintenance on MIC e.g. replace limit switch, 

tighten chain 

operator should check instruments frequently (twice per 

week) while construction is on-going 

low vol down for minimum of 5 days prior to audit; 

instrument had been unplug by contractor 
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EXPERIMENTAL LAKES AREA (E.L.A.) 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

ground cover: moss, rock and blueberry plants 

well removed from urban and agricultural infuences (remote 

site) 

representative of local area 

located at a research facility 

very little vehicular traffic in the area 

Disadvantages: 

on-site obstruction, tov/er 14 m NE 

incomplete windbreak - forest fire destroyed trees in 

surrounding area 

Instrument Evaluation 



MIC 



clutch adjustment too strong > 30 lbs 
motor humming when not operating 
• overall performance very good 
supported properly and level 

Storage Gauge 

wel 1 supported on wooden stand 

level 



I 
I 
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Operator Evaluation 

Operator: Ken Beatty 

operator feels sample handling area too cramped resulting 

in poor sample handling technique 

operator would like more information on APIOS as well as 

complete Technical and Operating Manual 

conscientious - checks sampler daily 

follows sample handling procedures 

Corrective Action Required 

move sampler away from tower 

perform minor maintenance on MIC - loosen clutch pad 
technologist to forward complete APIOS Technical and 
Operating Manual to operator 
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DORSET 

Site Evaluation 

Advantages: 

located at government research facility 

representative of topography and ground cover 

operated by APIOS regional technologist 

good accessibility 

CO- located at CAPMoN, CLIMAT sites 

no on-site obstructions 

no industrial or urban influences 

Disadvantages: 

proximity to gravel & sand road parking lot, however, a 
bank of trees separates the road from the sampler 

Instrument Evaluation 

Primary Sangamo 

clutch adjustment too strong > 35 lbs 
instrument in good working order 
well maintained 

Secondary Sangamo 

clutch adjustment too strong > 35 lbs 

limit switch requires adjustment - screw not making proper 

contact with switch 

motor runs or after instrument is turned off 

"C" clamp needs adjustment 

on/off switch not working properly 

hood drops suddenly 

good seal on knife edge 
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Primary Low Vol 

stand well supported and level 

sample line weathering - should be changed 

low vol in good housing 

operating properly 

slightly off calibration 

Secondary Low Vol 

stand well supported and level 
low vol in good housing 
operating properly 

Primary Storage Gauge 

orifice out of round 
well supported on stand 
not level 
"U" bolt not tight 

Secondary Storage Gauge 

orifice out of round 
well supported on stand 
not level 
"U" bolt not tight 

Operator Evaluation 

Operator: Cumulative Network - J. P. Varto 
Event Network - G. Carlyle 

Cumulative 

regional technologist 

very good attitude 

secondary MiC requires maintenance 

event operator needs refresher on sample handling 
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very little chance of sample contamination when sample 
changed 

Corrective Action Required 

event operator requires retraining 

- secondary MIC should be serviced 

- consideration should be given to paving parking lot and 
access road 
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WHITNEY 

Site Evaluation 

Advantages: 

representative of topography 

remote 

no agricultural or urban influences 

no vehicular activity in area 

no on-site obstruction 

accessible year round 

acceptable windbreak 

Disadvantages: 

proximity to sawmill 
proximity to swamp 
exposed earth 

Instrument Evaluation 

HIC 

clutch adjustment too strong 

limit switch needs to be adjusted - too much pressure on 

bracket 

overall performance good 

Storage Gauge 

well supported on wooden stand 

level 

mesh screen installed to keep birds out 
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Operator Evaluation 

Operator: Rick Hawkins 

operator not present at time of audit 

Corrective Action Required 

seed or sod to cover exposed earth 
perform regular maintenance on MIC 
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COLDWATER 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

no vehicular influences 

complete windbreak 

no on-site obstructions 

accessible year round 

Disadvantages: 

open pit limestone quarry 10 km E (blasting 2-3 times per 
week) 

Instrument Evaluation 

Sangamo Type A 

very good working order, no problems 

Storage Gauge 

well supported on wooden stand 
gauge orifice out of round 

Operator Evaluation 

Operator: Ron or Sharon Hancock 

operator does not follow sampling protocol 

folds over bag before it is tied 

does not force air out of bag before sealing 

does not always wear gloves when changing samples during 

winter months 
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sample bags not labelled 
does not wear gloves to clean gasket 
touches inside of bag with ungloved hands 
operators have an excellent attitude and are very enthus- 
iastic 

Corrective Action Required 

operators should be retrained and then checked on period- 
ically (every six months) 

historical data should be reviewed to see 1f the limestone 
quarry affects the integrity of the samples collected 
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GOWGANDA 

Site Evaluation 

Advantages: 

well removed from any major sources of pollution 

well removed from any agricultural or urban influences 

representative of local topography and ground cover 

representative of area 
no ground slope 

•J 

i 

Disadvantages: 1 



poor access during winter months 
tower (20 m SE) on-site obstruction 
poor windbreak 



MIC 



clutch set too strong > 35 lbs 
-■^ overall performance, good 

sample bucket slightly out of round 

Storage Gauge 

supported by a wooden stand 
contained transmission oil and water 
level 

Low Volume Sampler 

flow calibration not performed due to problems with the 

mass flow meter 

overall performance of low vol, good 

located in good shelter well supported 
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Operator Evaluation 

Operator: Bonnie Trajkowicz 

follows sample handling procedures outlined in APIOS 
Technical and Operating Manual 
operator feels she was not adequately trained 
over 1 year since procedures reviewed with operator 
operator should be retrained - even though she is follow- 
ing the outlined procedures, she is not confident about 
the quality of the samples she collects 
operator has an excellent attitude demonstrated by her 
concern for her sample handling technique 

Corrective Action Required 

retrain operator 

minor maintenance should be performed on MIC 
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WINISK 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

remote 

representative of area 

good access year round 

no on-site obstructions 

Disadvantages: 

snowmobile road adjacent to sampler 

sampler located at airport - possible contamination from 

airplane exhaust and human activity 

diesel generator 24 hrs/day 

Instrument Evaluation 

Sangamo Type A 

not operational when auditor arrived 
sensor grid broken 
clutch set too strong > 35 lbs 
possibly not functioning for months 

Storage Gauge 

storage gauge empty at time of audit 

no storage gauge results reported during months preceeding 

audit 
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Operator Evaluation 

Operator: Joe Bird 

does not change samples at proper time 
not careful with knife edge so as not to contaminate it 
sample bags never labelled 

operator performs no instrument checks, cleaning or main- 
tenance 

does not wear new gloves to pat down bag during 
installation 

operator claims he was never trained 
- operator claims he is not being paid and therefore has 
little interest 

operator has a very poor attitude 
submission sheets not filled out properly 
recoimend a new operator be found and properly trained 
since it is doubtful that present operator can be re- 
trained to operate the site properly 

Corrective Action Required 

move site away from snowmobile road or have snowmobilers 

use a different route 

replace operator or remove site 
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WHITMAN CREEK 

Site Evaluation 

Advantages: 

representative of topography and ground cover 

good access year round 

well removed from pollution sources and urban influences 

no on-site obstructions 

representative of local area 

Disadvantages: 

unreliable power source, numerous events missed during 

winter months 
proximity to road 

Instrument Evaluation 

Aerochem 

sensor sensitivity requires slight adjustment, sampler 

does not open for short duration (<5 minutes) events 

instrument in good working order 

better bag fit due to holes in sample bucket 

Standard Rain Gauge 

gauge level and has no cracks in it 
graduate requires cleaning 
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Operator Evaluation 

Operator: Sophie Angement 

follows all sample handling procedures outlined in APIOS 

Technical and Operating Manual 

excellent attitude and always informs technician of 

problems 

Corrective Action Required 

if a reliable power service cannot be obtained at this 
location the site should be moved to a location where 
reliable power is available, too many events missed 
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APPENDIX III 



"REPORTING LOW LEVEL DATA" BULLETIN 
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FROM THE QUALin CONTROL HANDBOOK FOR Revised March 13. 1980 

PILOT WATERSHED STUDIES, PLUARG, IJC J. L. Clark 



REPORTING LOW LEVEL DATA 

There are specific problems In the reporting of low level data which are 
•ssodated with the question: Is « substance present? While this question is 
seldom germane In IJC work concerned with loading estimation, It has so 
Influenced thinking about reporting low level data that It seems best to 
consider It in some depth before dealing with how such data are to be reported 
for IJC purposes. 

In wswering the question "Is a substance present?" there arc two 
possible correct conclusions which may be reached. One may conclude that the 
substance Is present when it is present, and one aay conclude that the 
substance Is not present* when it is not present. Conversely, there are two 
possible erroneous conclusions which may be reached. One may conclude that 
the substance is present when It is not, and one may conclude that the 
substance is not present when it Is. The first kind of error, finding 
sornething which is not there. Is called a TYPE I ERROR. The second kind of 
error, not finding something which Is there, Is called a TfPE II ERROR. 

These two types of errors are illustrated in the material that follows, 
using the result which might be obtained from a single analysis when the 
substance Is not present to illustrate Type I error and the inferences that 
might be drawn from a single analysis at two different actual concentrations 
to Illustrate Type II error. 

Of course Inferences as to water quality arc seldom, if ever, based on the 
ftsuU of a single analysis. A single result is used here to simplify the 
exposition. 

•Since Avogadro's number is very large, a pedant could argue that one should 
never claim that a substance is not present. A comon sense meaning of not 
present is Intended here, i.e. if measurement is being made in micrograms per 
litre the presence of a few nanograms per litre is irrelevant. 
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If the standard deviation, o, of *n ■nal/tical procedure has been 
determined at low concentrations Including 0, then the probability of making a 
Type I error can be set by choosing tn upproprlate a level to determine the 
Criterion of Detection. t 

for example, suppose that the standard deviation, o, of in analytical 
procedure is 6 yg/litre and that an o of 0.05 Is deemed acceptable to that 
the probability of making a Type I error is set at 5X. The Criterion of 
Detection can then be found from a table of cumulative norroal probabilities to 
be 1.645 - 1.645 X 6 vg/litre ■ 10 ug/lltre. 




Ary value observed below 10 yg/litre would be reported as less than the 
Criterion of Detection, since to report such a value otherwise would Increase 
the probability of making a Type I error beyond 5X. 

Note that the context of decision is the analytical result produced by the 

laboratory. A result Is obtained and a response made to It. Nothing has beer, 

said concerning the ability to detect a substance which is present at a 
specified concentration, 

KFTteFTofTof Detection nay be a new term to some. It refers to the minimum 
•lialytical result which must be observed before it can be stated that a 
substance has been discerned with an acceptable probability that the statement 
Is true. The terms Detection Limit or Limit of Detection are often used with 
this aeaning, but in this Handbook they are reserved for a more appropriate 
usage. 



'Z- 



Coreorcl St ii'iitiiic Cnr|>ir.itHin 



Once the Criterlwi of Oetectlor has been set. the probability of making a 
Type 11 error. 0. or Us complement 1-B, the probability of discerning the 
substance when it Is prtsent, can be determined for given true situations. 
(The probability 1-6 Is tometimes called the power of the test). 

Consider the same analytical procedure as above with a Criterion of 
Detection of 10 yg/litre. Suppose that the concentration of the sample 
being •ralyied Is 10 yg/litre, i.e. the concentration Is equal to the 
Criterion of Detection, If, all analytical results below the Criterion of 
Detection were reported as such, then the probability of discerning the 
substance would be 0.5 or 50%. 



Do not 

discern 



Discern 




P 



10 H9 



0.5 



1-/3=0.5 



Conversely, the probability of Baking a Type 11 error and failing to 
discern the substance would also be 0.5. From this example It can be seen 
that the probability of discerning a substance when Its concentration is equal 
to the Criterion of Detection is hardly overwhelming. In order for the 
probability of a Type II error to be equal to the probability of a Type I 
error. 8 • o. then the concentration of the sample being analyzed must be 
twice the Criterion of Detection. 
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This concentration of twice the Criterion of Detection Is the UmU of 
Detection when it has been decided that the risk of Baking a Type II error is 
to be equal to the risk of »aking • Type I error. 

The concept of Type H error has been emphasized because It Is usually 
Ignored. Generally, attention is paid to the avoidance of Type I error with 
no coisideraticn given to the probability of naklng a Type II error. It 
should also be recognized that when the probability of making a Type I error 
is decreased by selecting a lower a-level. the probability of wking a Type 
II error is increased. 

Having. It is hoped, made clear the conceptual context in which an 
o-level is set and the difference between the Criterion of Detection and the 
Limit of Detection. IJC requirements in the reporting of low level data can be 
considered. 

In oeneraJ, only under highly exceptional circumsttnces need 
there be a concern with avoiding Type 1 error when reporting 
data for IJC purposes. 

There are two reasons why Type 1 error is not a concern. First, tne IJC 
is not an enforcement agency, and therefore is not concerned that a single 
datum will lead it Into a false accusation that a substance is present when It 
is not. Second. In virtually all cases data are aggregated for IOC purposes 
In order to provide estimates of loadings and/or concentrations; therefore the 
avoidance of Type I error relates to data sets and not to the individual datum. 

This second point Is crucial. Rarely. If ever, will the analytical 
chemist have responsibility for inference from data sets or even be In a 
position to know which data «ay be combined. Therefore, censoring of results 
to prevent a possible faulty inference being drawn from an individual datum 
represents an unwarrented assumption of responsibility. 

In practice, these considerations mean that the Criterion of Detection may 
be set as low as possible. To state it another way. the o-level may be 

Ignored. 
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Oo tl» other t»ni, when reporting data for IJC purposes 
rt^ery effort miwt be mMde to avoid Type II error. 

The reason Is obvious. When results ire reported as "less than" or "below 
the Criterion of Detection," they are virtually useless for either estimating 
loadings or concentrations. 

In practice, this consideration teans that If a number can be obtained, it 
Is to be reported. 



codes' to be used in reporting low level data 

At Its April 12, 1976 neetlng the Data Quality Sub conn it tee of the Water 
Quality Board passed a resolution that 2 new codes be made available In data 
storage systems for remarks concerning data used In IJC reports. The codes 
are T and U. 

The T code has the following meaning: "Value reported Is less than 
Criterion of Detection.* The use of this code warns the data user that the 
Individual datum with which It Is associated does not. In the Judgement of the 
laboratory which did the analysis, differ significantly from 0. 

It should be recognized than an Implied significance test which fails to 
reject the null hypothesis that a result does not differ from a standard value 
In no way diminishes the value of the result as an estimate. To Illustrate: 
a result of 9 yg on a test whose o ■ 6 yg can not be regarded as 
significantly different from for any o-level less than 0.D67; however. If 
a significance test were made with o • 0.1. then the null hypothesis would 
be rejected and the result deemed significantly different from 0. 

So the result, 9 >ig. could be reported as "Below the Criterion of 
Detection" for all o less than 0.067 and could be reported as simply '9 yg" 
for all a greater than 0.067. But however reported, the result of 9 yg 
remains the best estimate of the true value since changing the risk of making 
a Type I error neither augments or diminishes the value of an estimate. 
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It Bay b€ added that low level results ire better estimates, In the sense 
of being ■ore precise, than higher results $ince for all analytical tests with 
which Me are acquainted the standard deviation of the test Increases with the 
concentration. 

The W code has the following meaning: 'Value observed Is less than lowest 
value reportable under T code." This code is used when a positive ?alue is 
not observed or calculated for a result. In these cases the lowest reportable 
value, which is the lowest positive value which Is observable, is reported 
ifith the W. 

The following example Illustrates the use of the codes: 

Suppose that a laboratory has determined that Us Criterion of Detection 
for total phosphorus Is 10 yg/litre, and suppose In addition that the 
smallest increment that can be read on the analytical device corresponds to a 
concentration of 2 pg/litre. Given these conditions, any value observed 
>10 yg/litre would be reported without an accompanying code; any value 
observed >2 pg and <10 yg would be reported with the T code; If no 
Instrument response were observed, the result would be reported as 2H. 



REPORTING NEGATIVE RESULTS 

Mith many analytical procedures there will always be an instrument 
response, so the W code will not apply. In particular, this lack of 
applicability will occur when a result Is obtained through subtraction of a 
blank correction. In this case negative results will often be obtained; in 
fact. If the constituent of Interest is not present, one would expect negative 
results to occur as often as positive. 

In order that valid inferences may be wde from surveillance data. It is 
iBfJortant that negative results be reported as such. Consider the following 
three different ways of reporting the same results. The left hand colunr 
lives results In a heavily censored form; the center colunr has negative 
results censored; the right hand coliinD gives the results as obtained. 
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<3 yg 2 wg 2 VS 

<3 -2 

<3 -1 

4 4 4 

3 3 3 

<3 *3 

<3 1 I 

<3 -1 

<3 & 

<3 2 2 

Nothing can be done with the results In the left hand column except to 
conclude that we don't know whether the constituent Is present or not; the 
sampling and analytical effort have been wasted. 

If the results In the center column were taken at face value, one could 
conclude that the mean concentration was 1.2 yg with a standard error of the 
mean of 0.467 and 95X confidence limits for the mean of 0.14 yg and 2.26 yg. 
Since the confidence limits do not Include lero. It would appear that the 
evidence supports the presence of the constituent. 

Analysis of the uncensored results of the right hand colunr gives a mean 
concentration of 0.5 ug. a standard error of the mean of 0.719. and 951 
confidence limits for the mean of -1.13 yg and 2.13 yg. The correct 
conclusion can be drawn that the evidence Is Insufficient to support the 
presence of the constituent. 

Note that the censored data of the center coluitr distort both the mean and 
the standard error of the data, making the data appear more precise than they 
•re. 

Of course any result of or less which ts reported should be reported 
with the T code. 
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